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PREFACE 





This book introduces applied numerical methods for engineering and Science 
students in Sophomore to Senior levels; it targets the students of today who do 
not like or do not have time to derive and prove mathematical results. It can 
also Serve as a reference to MAILAB applications for professional engineers 
and Scientists，Ssince many of the MAILILAB codes presented after introducing 
each algorithm'"s basic ldeas can easily be modified to solve Similar problems 
even by those who do not know what is going on inslde the MAILAB routines 
and the algorithms they use. Just as most drivers only have to know where to 
go and how to drive a car to get to their destinations，most users only have to 
know how to define the problems they want to solve using MAILAB and how 
to use the corresponding routines to Solve their problems. We never deny that 
detailed knowledge about the algorithm (engine) of the program (car) is helpful 
for getting safely to the solution (destination); we only Imply that one-time users 
of any MAILAB program or routine may use this book as well as the students 
who want to understand the underlying principle of each algorithm. 

In this book，we focus on understanding the fundamental mathematical con- 
cepts and mastering problem-solving Skills using numerical methods with the 
help of MAILAB and skip Some tedious derivations. Obviously，basic con- 
cepts must be taught so that students can properly formulate the mathematics 
problems. Afterwards，students can directly use the MAILAB codes to solve 
practical problems. Almost every algorithm introduced in this book is followed 
by example MAILAB code with a friendly interface so that students can easily 
modify the code to solve real life problems. The selection of exercises fol- 
lows the Some philosophy of making the learning easy and Practical. Students 
Should be able to solve Similar problems immediately after taking the class using 
the MAILAB codes we provide. For most students 一 and particularly nonmath 
majors 一 understanding how to use numerical tools correctly in solving their 
problems of interest ls more Important than studying lengthy proofs and deriva- 
tions， 

MAILAB is one of the most developed software packages available today. 
It provides many numerical methods and it ls very easy to use, even for people 
without prior programming experience. We have Supplemented MATILAB's built- 
in functions with more than 100 small MAILAB routines. Readers Should find 
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these routines handy and useful. Some of these routines give better results for 
Some problems than the built-in functions. Students are encouraged to develop 
their own routines following the examples. 

The knowledge in this book is derived from the work of many eminent Sci- 
entists，Scholars, researchers, and MAILAB developers, all of whom we thank. 
We thank our colleagues，students,，relatives，and friends for their Support and 
encouragement. We thank the reviewers，whose comments were So helpful in 
tuning this book. We especially thank Senior Researcher Yong-Suk Park for his 
invaluable help in correction. We thank the editorial and production Staff of John 
Wiley 久 Sons，Inc. including Editor Val Moliere and Production Editor Lisa 
VanHorn for their kind, effticient, and encouraging guide. 


WON YOUNG YANG 
WENWU CAO 
TAE-SANG CHUNG 
JoHN MORRIS 
Octoper 2004 


MATLAB USAGE AND 
COMPUTATIONAL ERRORs 





1.1 BASIC OPERATIONS OF MATLAB 


MAILAB is a high-level software package with many built-in functions that 
make the learning of numerical methods much easier and more interesting. In 
thlis Section we will introduce Some basic operations that will enable you to 
learn the software and build your own programs for problem solving. In the 
Workstation environment，you type“matlab”to start the program，while in the 
PC environment, you Simply double-click the MAILAB icon， 

Once you start the MAILAB program, a Command window will open with the 
MAILAB prompt >>. On the command line, you can type MAILAB commands， 
functions together with their input/output arguments, and the names of Script files 
containing a block of statements to be executed at a time or functions defined 
by users. The MAILAB program files must have the extension name *** .m to 
be executed in the MAILAB environment, I you want to create a new Mr-file 
Or edit an existing file, you click File/New/M-file or File/Open in the top left 
corner of the main menu, fnd/selecUVload the file by double-clicking it, and then 
begin editing it in the Editor window. If the path of the file you want to run 
is not listed in the MAILAB search path, the fle name will not be recognized 
by MATLAB. In such cases,， you need to add the path to the MAILAB-path 
list by clicking the menu “File/Set_Path” in the Command window, clicking the 
“Add_Folder button, browsing/clicking the folder name, and finally clicking the 
SAVE button and the Close button. The /ookfor command ls available to help 
you find the MAILAB commands/functions which are related with a job you 
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want to be done. The jpeiP command helps you know the usage of a particular 
command/function. You may type directly in the Command window 


>>J0okfor repeat OFT >>heJp for 


to find the MAILAB commands in connection with “repeat” or to obtain infor- 
mation about the“for loop”. 


1.1.1 Input/Output of Data from MATLAB Command Window 


MAILAB remembers all input data in a session (anything entered through direct 
keyboard input or running a Script fle) until the command “clear() ”is given OF 
you exit MAILAB. 

One of the many features of MAILAB is that it enables us to deal with the 
Vectors/matrices in the Same way as Scalars. For instance, to input the matri- 
Ces/vectors， 





3 
1 2 3 
4 5 | 妃 一 2 ， C=[l -2 3 -4] 


type in the MAILAB Command window as below: 


>>A= [123;456] 
A=1 2 3 
4 5 6 
[3;-2;1]; %put the Semicolon at the end of the statement to Suppress 
the result printout onto the Screen 
>>C= [1 -2 3 -4] 


V 
V 

四 
册 


At the end of the statement，Ppress <Enter> 寺 you want to check the result 
of executing the statement immediately. Otherwise, type a Semicolon“; ”before 
pressing <Enter> So that your window will not be overloaded by a long display 
of results. 


1.1.2 Input/Output of Data Through Files 


MAILAB can handle two types of data files. One is the binary format mat- 
fles named #*##.mat. This kind of fle can preserve the values of more than one 
variable, but will be handled only in the MAILAB environment and cannot be 
Shared with other programming environments. The other is the ASCJI dat-files 
named ***.dat,， which can be Shared with other programming environments, but 
Preserve the values of only one variable. 

Below are a few sample statements for storing Some data into a mat-file in 
the current directory and reading the data back from the mat-file: 


>>Save ABC AB Cs%store the values of A;,B,C into the file "ABC.mat' 
>>CJear A C %clear the memory of MATLAB about A,C 
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>>A %wWhat is the Value of A? 
?3? Undefined function or variable 'A- 
>>10ad ABC A C s%read the values of A;C from the file ' ABC.mat' 
>>A %the Value of A 
A=1 2 3 
4 5 6 


HI you want to store the data into an ASCII dat-file (in the current directory)， 
make the filename the Same as the name of the data and type“/ascii” at the 
end of the saye statement， 


>>Save B.dat B /ascii 


However，with the Save/load commands into/from a dat-file, the value of only 
one variable having the lowercase name can be Saved/loaded, a scalar or a vec- 
tormatrix. Besides, non-numeric data cannot be handled by using a dat-file. 下 
you Save a String data into a dat-file, its ASCJHI code will be Saved. I a dat-file 
is constructed to have a data matrix in other environments than MAILAB, every 
line (row) of the file must have the Same number of columns. If you want to read 
the data from the datrfile in MAILAB, just type the (lowercase) filename ***.dat 
after 1oad`， which will also be recognized as the name of the data contained in 
the dat-file. 


>>1oad b.dat s%read the value of variable b from the ascii file 'b.dat， 


On the MAILAB command line, you can type “nm112” to run the following 
Mr-fle “nm112.m” conslsting of several file input(save)/output(load) statements. 
Then you will see the effects of the individual statements from the running 
results appearing on the Screen. 


%nm112 .m 

Clear 

A= [123;456] 

B = [3;-2;1]; 

C(2) = 2; C(4) = 4 

disp('Press any key to see the input/output through Files ') 
Save ABC AB Cs%save A,B &C as a MAT-file named '` ABC.mat- 
Clear('A'，C') s#%remove the memory about A and C 

1Load ABC A C %read MAT-file to recollect the memory about A and C 
Save B.dat B /ascii %save B as an ASCII-file named 'b.dat' 
Clear B 

1Load b.dat s%read ASCII-file to recollect the memory about b 


input( Enter Xx: ) 
format Short e 


X 
format rat，X 

format Jong，xX 
format Short，X 
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1.1.3 Input/Output of Data Using Keyboard 


The command “Input”enables the user to input Some data via the keyboard. 
For example， 


>>X = input(' Enter X: ，) 
Enter x: 1/3 
x = 0.3333 


Note that the fraction 1/3 is a nonterminating decimal number, but only four 
digits after the decimal point are displayed as the result of executing the above 
command. This 18 a _ choice of formatting in MAILILAB. One may choose to 
display more decimal places by using the command "format'`，which can make 
a fraction show up as a fraction, as a decimal number with more diglits，or even 
in an exponential form of a normalized number times 10 to the power of some 
integer. For instance: 


>>format rat *#%as a _ rational number 


>>X 

Xx = 1/3 
>>format Jong %5as a decimal number with 14 digits 
>>X 


x = 0.33333333333333 
>>format Jong e %as a 1ong exponential form 
>>X 


Xx = 3.333333333333333e-001 


>>format hex %as a hexadecimal form as represented/stored in memory 
>>X 


X = 3fd5555555555555 


>>format Short e %as a Short exponential form 


x = 3.3333e-001 


>>format Short %back to a short form (default) 
>>X 


Xx = 0.3333 


Note that the number of displayed digits is not the actual number of significant 
digits of the value stored in computer memory. This point will be made clear in 
Section 1.2.1. 

There are other ways of displaying the value of a variable and a string on the 
Screen than typing the name of the variable. Two useful commands are “disp()” 
and 'fprintf() .The former displays the value of a variable or a String without 
XxX= or'ans = ;thelatterdisplaysthe values of several variables in a specified 
format and with explanatory/cosmetic Strings. For example: 


>>disp('The value of X= ')，,dqisp(x) 
s%5disp(' string_to_ display'” or variable_name) 
The value of Xx = 0.3333 


Table 1.1 summarizes the type specifiers and Special characters that are used in 
“fprintf()” Statements. 

Below is a program that uses the command “Input” So that the user could 
input Some data via the keyboard. 玉 we run the program, it gets a value of the 
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Table 1.1 Type Specifiers and Special Characters Used in fprintf() Statements 


Character type 
String type 


Decimal integer number type 

Floating point number type CR return 
Decimal exponential type Form feed 
Hexadecimal integer number 5 

Floating number in 16 hexadecimal digits(64 bits) 





temperature in Fahrenheit [FE] via the keyboard from the user，converts it into 
the temperature in Centigrade [CC] and then prints the results with some remarks 
both onto the Screen and into a data file named nml13.dat . 


%nm113 .m 
input(' Input the temperature in Fahrenheit[F]: ); 
C = 5/9x*(f-32); 
fprintf( '%55.2f(in Fahrenheit) is #%55.2f(in Centigrade).N\n' ,fc) 


fid=fopen( nm113.dat' ，w' ); 
fprintf(fid， '%5.2f(Fahrenheit) is %5.2f(Centigrade).\n' ,fc) 
fclose(fid) ; 





In case you want the keyboard input to be recognized as a string, you Should 
add the character 'S' as the second input argument. 


>>ans = input('Answer <yes> or <no>: '，'S') 


1.1.4 2-D Graphic Input/Output 


How do we plot the value(S) of a vector or an array? Suppose that data reflecting 
the highestlowest temperatures for 3 daygs are Stored asa3 x 2array in an ASCII 
fle named “temp .dat . 

The job of the MAILAB program “nm114_1.m” is to plot these data. Running 
the program yields the graph shown in Fig. 1.1a. Note that the first line ls a 
comment about the name and the functional objective of the program(file), and 
the fourth and fifth lines are auxiliary statements that designate the graph title 
and units of the verticalhorizontal axis; only the Second 儿 third lines are indis- 
pensable in drawing the colored graph. We need only a few MAILAB statements 
for this artwork，which shows the power of MAITLAB. 


%nm114 1: plot the data of a 5Xx2 array Stored in "temp .dat" 
1Load temp .dat 
clf，plot(temp) %clear any existent figure and plot 


tit1le('the highest/]Iowest temperature of these days ' ) 
ylLabel( degrees[C]' )，Xxlabel('day ') 
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The highestlowest temperature of days The highestlowest temperature of days 
25 25 
说 20 D 20 
本 从 
15 15 
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瑟 10 上 上 ] 羡 10 | 
5 | | | 5 | | | 
1 2 3 4 day 5 11 ”12 14 day 16 17 
(a) Domain of the horizontal (b) Domain of the horizontal 
variable unspecified variable specified 


Figure 1.1 Plot ofa5 x2matrix data representing the highest/lowest temperature. 


Here are several things to keep in mind. 


。 The command plot () reads along the columns of the 3 x 2 array data given 
as its input argument and recognizes each column as the value of a vector. 

。 MAILILAB assumes the domain of the horizontal variable to be [1 2 .. 9] by 
default, where 9 equals the length of the vector to be plotted (See Fig. 1.1a). 

。 The graph is constructed by connecting the data points with the Straight lines 
and ls piecewise-linear，while it looks like a curve as the data points are 
densely collected. Note that the graph can be plotted as points in various 
forms according to the optional input argument described in Table 1.2. 


(Q1H) Suppose the data in the array named "temp” are the highesUlowest temperatures 
measured on the 11th,12th,14th,16th, and 17th days, respectively. How should we 
modify the above program to have the actual days Shown on the horizontal axis? 


(Al) Just make the day vector [11 12 14 16 17] and use it as the first input argument 
of the plot() command. 


>>days = [11 12 14 16 17] 
>>p1lot(days,temp) 


Executing these statements，we obtain the graph in Fig. 1.1b. 


(Q2) What statements Should be added to change the ranges of the horizontal/vertical 
axes into 10-20 and 0-30, respectively, and draw the grid on the graph2 


Table 1.2 Graphic Line Specifications Used in the plot() Command 


Line Type Point Type (Marker Symbol) 


- Solid line * (asterisk) m : magenta 


dotted line 0 (circle) : y : yellow 
-- dashed line : x : X-mark c : Cyan (Sky blue) 
-. dash-dot S : 
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(A2) >>axis([10 20 0 30])，grid on 
>>p1lot(daystemp) 

(Q3) How do we make the scales of the horizontal/vertical axes equal so that a circle 
appears round, not like an ellipse? 

(A3) >>axis('equal') 

(Q4) How do we have another graph overlapped onto an existing graph? 

(A4) Iyou use the “ho1d on” command after plotting the first graph, any following 


graphs in the Same Section will be overlapped onto the existing one(S) Tather 
than plotted newly. For example: 


>>hold on，plot(days,temp(:),，1)，bx*'，daysitemp(:;，2)，ro') 


This will be good until you lssue the command hold off” or clear all the graphs 
in the graphic window by using the“clf” command. 


Sometimes we need to see the interrelationship between two variables. Sup- 
pose we want to plot the lowesthighest temperature，respectively，along the 
horizontal/vertical axis in order to grasp the relationshijp between them. Let us 
try using the following command: 


>>p1lLot(temp(: ,1) ,temp(:，2)，Kkx') 55 temp(:,， 2) vS. temp(:,1) in black  X- 


This will produce a pointwise graph，which is fine. But, it you replace the third 
input argument by “b: orjust omit it to draw a piecewise-linear graph connecting 
the data points as Fig. 1.2a, the graphic result looks clumsy, because the data on 
the horizontal axis are not arranged in ascending or descending order. The graph 
will look better if you Sort the data on the horizontal axis and also the data on 
the vertical axis accordingly and then plot the relationship in the piecewise-linear 
style by typing the MAILAB commands as follows: 


>>[temp1,I] = Sort(temp(:,1)); temp2 = temp(I,2); 
>>p1Lot(temp1,temp2) 


The graph obtained by using these commands is Shown in Fig.1.2b, which looks 
more informative than Fig.1.2a. 








25 


20 


15 














10 
4 6 8 10 12 4 6 8 10 12 


(a) Data not arranged (b) Data arranged along the horizontal axis. 


Figure 1.2 Examples of graphs obtained using the plot ( ) command. 
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We can also use the plot() command to draw a circle. 


>>P = 1; = [0:0.01:2]*pi; 5 [0:0.01:2] makes [0 0.01 0.02 .，2] 
3 ws h),rx*sin(th)) 
>>p1lot(rx*exp(j*th)) %5alternatively， 


Note that the plot() command with a Sequence of complex numbers as its first 
input argument plots the real/imaginary parts along the horizontal/vertical axis. 

The polar() command plots the phase (in radians)/magnitude given as its 
firstsecond input argument, reSpectively (See Fig.1.3a). 


>>polar(th,exp(-th)) %polar plot of a Spiral 


Several other plotting commands, such as semilogx(), semilogy()，1og1og ()， 
stairs()，stem()，bar()/barh(),， and hist(),，may be used to draw various 
graphs (Shown in Figs.1.3 and 1.4). Readers may use the “he1Lp” command to get 
the detailed usage of each one and try running the following MAILAB program 
mml14 2.m ”. 


snm114 2: plot Several types of graph 
= [0: .02:1]*piI; 

Subp1lot(221)，polar(th,exp(- 
Subp1lot(222)，Semi1logx(exp(t 
Subp1lot(223)，sSsemi1logy(exp(t 
Subplot(224)，1og1log(exp(th) 
pause，cCclf 

Subp1lot(221 
Subp1ot(222 


th ) ) 
h) ) 
h) ) 
) 


Stairs([ 


1320]) 
stem([1 3 2 0 
4 5 


0 
) 
) 
] 


) ， 

) ， 
Subplot(223)，bar([2 3; 

) ， 


] 
] 
Subp1Lot(224 barh([2 3; 4 5 
pause，cCclf 
= [0.3 0.9 1.6 2.7 3 2.4]; 
Subplot(221)，hist(y;,3) 


Subplot(222)，hist(y;，0.5 + [0 1 2]) 


) 





Moreover, the commands sprintf(), text(),，and gtext() are used for com- 
bining Supplementary statements with the value(S) of one or more variables to 
construct a_ string and printing it at a_ certain location on the existing graph. 
For instance，let us try the following statements in the MAILAB Command 
window: 


>>f = 1./[1:10]; plot(f) 

>>n = 3j [serrmsgl = Sprintf('f(#%s1d) = %55.2f' nfln)) 

>>text(3,f(3)，,SsS) %writes the text String at the point (3,f(3)) 

>>gtext( f(x) = 1/Xx') %writes the input string at point clLicked by mouse 


The command ginput() allows you to obtain the coordinates of a point 
by clicking the mouse button on the existent graph. Let us try the following 
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Figure 1.3 Graphs drawn by various graphic commands. 
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(e) hist (0.3 .. 2.4], 3) (f) hist ([.], [0.5 1.5 2.5]) 


Figure 1.4 _ Graphs drawn by various graphic commands. 
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commands: 

>>[X,yibutkey] = ginput %#%get the xy coordinates & # of the mouse button 
or ascii code of the key pressed till pressing the ENTER Key 

>>[Xx,yibutkey] = ginput(n) #%srepeat the Same job for up to n points clLicked 


1.1.5 3-D Graphic Output 


MAILAB has several 3-D graphic plotting commands such as plot3(), mesh()， 
and contour(). plot3() plots a 2-D valued-function of a scalar-valued vari- 
able; mesh()/contour() plots a scalar valued-function of a 2-D variable in a 
Imesh/contour-like style, respectively， 

Readers are Ifecommended to use the help command for detailed usage of each 
command. Try running the MAILAB program “nmll13.m” to See what fgures 
will appear (Figs.1.53 and 1.0). 


s%nm115: to plot 3D graphs 

0:pi/50:6xpii 
expt = exp(-0.1x*t) ; 
xt = expt.*cos(t); yt = expt.*Sin( 七 ) ; 
s%5dividing the Screen into 2 X 2 Sections 
Subplot(221)，plot3(xt，yt， ，grid on %he1ix 
Subp1ot(222 ) ， ea ，grid on，Vview([0 0 1]) 
Subplot(223)，plot3(t grid on，Vview([1 -3 1]) 
Subp1lot(224) ， plot3(t， ，grid on，Vview([0 -3 0]) 
pause，cClf 
X= -2:.1:2) Vy= -2: 
[X,Y] = meshgrid(xy,y 
Subp1lot(221)，mesh(X 
Subp1lot(222)，mesh(X 
pause，Vview([30,30] ) 
Subp1lot(223)，contour( 
Subp1lot(224)，contour( 


t) 
t) 
yt) ， 
xt) 


站 二 2 

于 区 人 人 十 Ye 25 
Y,Z)，grid on 5%5[azimuth,elevation] = [-37.5,30] 
Y，Z 


) ; 
LVY， )，Vview([0,20])，grid on 





1.1.6 ”Mathematical Functions 


Mathematical functions and Special reserved constants/variables defined in MATI- 
LAB are listed in Table 1.3. 

MATLLAB also allows us to define our own function and store it in a file 
named after the function name So that it can be used as 夺 it were a built-in 
function. For instance,， we can define a scalar-valued function: 


户 Co) = 1/G +8x2) 


and a vector-valued function 


| AGCxz)| xz 十 4x2 一 5 
人 [种 | 加 医 一 2xl 一 3x2 一 2.5 
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(a) plot3(cos(t), sin(t), t) (b) plot3( ), view([0 0 1]) 
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(c) plot3( ), view [(1 -3 1)] (d) plot3( ), view ([0 -3 0]) 


Figure 1.5 Graphs drawn by the plot3() command with different views. 
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(a) mesh( ), view (-37.5, 30) (b) mesh( ), view (30, 20) 
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(c) contour(X,Y,Z) (d) contour(X,Y,Z, [0.5, 2, 4.5]) 


Figure 1.6 Graphs drawn by the mesh() and contour () commands. 


as follows. 


function y = f1(x) function y = f49(X) 


yY=1./(1+8*Xx.^2) y(1) = X(1)*x(1)+4*Xx(2)*x(2) -5; 
y(2) = 2*Xx(1)*x(1)-2*x(1)-3*x(2) -2.5; 
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Table 1.3 ” Functions and Variables Inside MATLAB 


CI 
[CT 
CI 


number [rad] 
) 


tanh (X (ex< 一 e)/(e 十 e) round(Xx) The nearest integer 


(round-off) 


acosh(X) The nearest integer 
toward 0 

asinh(X) sinh-1(x) X The greatest integer 
必 

atanh(X) tanh-1(x) ceil(Xx) The smallest integer 
之 郑 

maX Maximum and its index Sign(Xx) 1(positive)/O/- 
lnegative) 

In 


Sort in the ascending polyval Value of a polynomial 
Order function 


oo 
Toots 
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Table 1.3 (continueoa) 


JIndex of element(s) Roots of polynomial 
flops(0) Reset the fops count to 
Zero 


f1ops Cumulative # of foating toc Read the stopwatch 
point operations timer (elapsed time 
(unavailable in from tic) 
MAILAB 6.x) 


Reseved Variaplies with Special Mea1i18 


亏 


Start a Stopwatch timer 


point relative accuracy positive number 


Exit while/for loop Largest number (co) 
end The end of for-loop or Not a_Number 
直 while，case Statement (undetermined) 
or an array index 
nargin Number of input nargout Number of output 
arguments arguments 
varargin Variable input argument varargout Variable output 
list argument list 


Machine epsilon foating realmax realmin Largest/smallest 





Once we Store these functions into the fles named “fl.m” and 人 9.m” after the 
function names, respectively, we can call and use them as needed inside another 
M-file or in the MAITLAB Command window. 


>>f1([0 1]) %5several values of a scalar function of a Scalar variabl1e 
ans = 1.0000 0.1111 


>>f49([0 1]) %a value of a 2-D vector function of a vector variab1le 
ans = -1.0000 -5.5000 


>>feval(' fl1'，[0 1])，feval('f49 ，[0 1]) %equivalentlLy，yields the Same 
ans = 1.0000 0.1111 
ans = -1.0000 -5.5000 


(Q5) With the function fl(x) deftined as a scalar function of a scalar variable， we enterT 
avector as its input argument to obtain a seemingly vector-valued output. What's 
going on? 
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(A5) Itis just a set of function values [fl(xl) fl(x2) ...] obtained at a time for Several 
values [X1 x2...] of x. In expectation of one-Sshot multi-operation, lt is a good 
practice to put a dot(.) just before the arithmetic operators *(multiplication)， 
/(division)，and ^ (power) in the function definition So that the term-by-term 
(termwise) operation can be done any time. 


Note that we can define a simple function not only in an Independent M-file， 
but also inside a program by using the inline() command or just in a form of 
literal expression that can be evaluated by the command eval(). 


>>f1 = inline(' 1./(1+8*Xx.^2) ，X' ) 

>>f1([0 1])，feval(f1,[0 1]) 
ans = 1.0000 0.1111 
ans = 1.0000 0.1111 

>>f1 = 1./(1+8*x.^2) ;)X= [0 1]; eval(fl) 
ans = 1.0000 0.1111 


As far as a polynomial function ls concerned, it can Simply be deftined as its 
coefficient vector arranged in descending order. It may be called to yield its 
value for certain value(Ss) of its Independent variable by using the _ command 
poJyval( ). 


>>p = [10 -32]; %polynomial function DOD=X3 一 3x 二 2 
>>polyval(p,[0 1]) 
ans = 2.0000 0.0000 


The multiplication of two polynomials can be performed by taking the con- 
volution of thelir coefficient vectors representing the polynomials in MAILAB， 
Since 


(CNwXX 十 :十 ax 十 a0)(CNXY 十 :十 Dixz 十 Do) 一 cx 十 :十 cixz 十 co 


where 


min(K,N) 
cf 一 》， a_np，， fork==2NV2N--1... ,10 
1 一 max(0,K 一 N) 


This operation can be done by using the MAILAB built-in command conv () as 
jllustrated below. 


>>8 [1 -1]; b=[11 1];c= conv(a;b) 


1 0 0 -1s%meaning that (x lD)C2 +x+HD=xz3 二 0.x2 二 0. 1 





C 


But，in case you want to multiply a polynomial by only x"，you can Simply 
append 7 zeros to the right end of the polynomial coefficient vector to extend 
its dimension. 


0] s%5equivalently，CcC = conv(a,[1 0 0]) 
meaning that (CC2 十 2x 十 3)xz2 = x4+2r3 十 3x2 十 0.x 十 0 





0 
5 
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1.1.7 ” Operations on Vectors and Matrices 


We can define a new Scalarvectormatrix or redefine any exlsting ones in terms 
of the existent ones or irrespective of them. In the MAILAB Command window， 
let us define4 and 已 as 


3 
1 2 3 
| 5 | 2 六 


by typing 
>>A= [123;456],，B= [3;-2;1] 


We can modify them or take a portion of them. For example: 


>>A = [A;7 8 9] 


A=1 2 3 
4 5 6 
7 8 9 
>>B= [B [10-1]] 
B = 3 1 
-2 0 
1 -1 


Here, the apostrophe (Prime) operator (') takes the complex conjugate transpose 
and functions virtually as a transpose operator for real-valued matrices. I you 
want to take just the transpose of a _ complex-valued matrix，you Should put a 
dot(.) before ', that 1Ss“.…. 

When extending an existing matrix or defining another one based on it the 
compatibility of dimensions should be observed. For instance, if you try to annex 
a4xlmatrxinto the 3x1lmatrxB, MALILAB will reject it squarely，giving 
you an error messSage. 
>>B = [B ones(4,1)] 

?3?3?A11 matrices on a row in the bracketed expression must have 

the Same number of rows 


We can modify or refer to a portion of a given matrix. 


>>A(3,3) = 0 
A=1 2 3 
4 5 6 
7 8 0 
>>A(2:3,1:2) %from 2nd row to 3rd row，from 1st column to 2nd column 
ans = 4 5 
7 8 


>>A(2,:) %52nd row，all columns 
ans = 4 5 6 


The colon (:) is used for deftining an arithmetic (equal difference) Sequence 
without the bracket [] as 


>>t = 0:0.1:2 
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which makes 
t= [0.00.10.2 ... 1.9 2.0] 


(Q6) What 直 we omit the increment between the leftright boundary numbers? 
(A6) By default, the increment is 1. 


>> 七 


0:2 
t=01 


2 


(Q7) What ithe right boundary number is Smaller/greater than the left boundary 
number with a positive/negative increment2 


(A7) It yields an empty matrix，which is useless. 


>> 七 
七 


0:-2 
Empty matrix: 1-by-0 


(Q8) I we define just Some elements of a vector not fully, but Sporadically，will we 
have a row vector or a column vector and how will it be flled in between'” 


(A8) We will have a row vector filled with zeros between the defined elements. 


>>D(2) = 2; D(4) = 3 


(Q9) How do we make a column vector in the Same Style? 
(A9) We mnust initialize it as a (Zero-filled) row vector, Prior to giving it a value. 


>>D = zeros(4,1); D(2) = 2; D(4) = 3 
D = 0 
2 
0 
3 


(Q10) What happens ii the Specified element index of an array exceeds the defined 
Tange7 
(Al0) It is rejected. MAITLAB does not accept nonpositive or noninteger indices. 


>>D(5) 
?3?? Index exceeds matrix dimensions ， 
>>D(0) = 1; 
?3?? Index into matrix Is negative or zero， 
>>D(1.2) 
?3?? Subscript indices must either be real positive 
integers ,， 


(Q11) How do we know the size (the numbers of rowSs/columns) of an already- 
defined array? 
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(All) Use the length() and size() commands as indicated below. 


>>Jength 
ans 


(D 
>>[M,N] = Size(A) 
M 3 

3 


N 


MAILAB enables us to handle vectormatrix operations in almost the Same 


Way 


as Scalar operations. However, we must make Sure of the dimensional com- 


patibjlity between vectors/matrices，and we mnust put a dot (.) in front of the 
operator for termwise (element-by-element) operations. The addition of a matrix 
and a scalar adds the scalar to every element of the matrix. The multiplication 


of a 


matrix by a scalar multiplies every element of the matrix by the Scalar. 


There are several things to know about the matrix division and inverslon. 


了 R 了 emark 1.1. Rules of VectorMatrix Operation 


1. 


FEor a matrix to be invertible, it must be Square and nonsingular; that ls, the 
numbers of its rows and columns must be equal and its determinant must 
not be Zer0. 


. The MAILAB command pinv(A) provides us with a matrix X of the Same 


dimension as 47 such that 4X4 一 4 and X4X = X. We can use this 
command to get the rightleft pseudo- (generalized) inverse 47[4A4Z]-L/ 
[474]-147 for a matrix 4 given as its input argument，depending on 
whether the number (AM) of rows is Smaller or greater than the number 
(NV) of columns, so long as the matrix ls of full rank; that is, rank(4) 一 
min(M, NW)[K-1, Section 6.4]. Note that 47 [447]-17[474]-147 is called 
the rightUleft inverse because it is multiplied onto the TighUleft Side of 4 
to yield an identity matrix. 


. You should be careful when using the pinv(A) command for a rankc- 


deficient matrix, because its output ls no longer the TighUleft inverse, which 
does not even exist for rank-deficient matrices. 


. The value of a scalar function having an array value as its argument is also 


an array with the Same dimenslon. 


Suppose wehave definedvectors al, qz, pl, pz andmatrices 41, 4?, Basfollows: 


>>al 


>>A1 


= [-123];a2= [452];bl= [1-3]';b2= [-20]; 


al 一 [一 1 2 3 ]， az 一 [4 5 分 | E| 写 | 2D2 一 [一 ] 2 3] 


= [al;ja2]，A2 = [al;[b2 1]]，B = [bl b2'] 


1 2 3 1 2 3 1 _2 
4=| 4 5 让 4=| 卫 0 中 :| 攻 ， 同 
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results of various operations on these vectors/matrices are as follows (pay 


attention to the error message): 


>>A3 


>>AB 


公 
工 


=Al+A2，A4=A - A2， 1+A s%matrix/scalar addition/subtraction 
AS3=-246 A4=0 0 0 ans=0 3 4 
2 5 3 6 5 1 5 6 3 


= Alx*B % 4B(1 ,11) 一 》 41(00, 人 )B(K 7) matrix multiplication? 
大 


33? Error USing ==> * 
nner matrix dimensions must agree . 


>>BA1 = B*A1 5 regular matrix multip1lication 


BA 


>>AA 


AA 


>>AB 
名 
M 


>>A1 
Al1 


>>A1 
a 


>>A5 
>>A5 
A5 


>>A5 


>>A1 


1=-9 -8 -1 
3 -6 -9 
= A1.*A2 %termwise multiplication 
= 1 4 9 
-8 0 2 
=A1.*B 5%5 AB8(1,n) = Ai(1,m)B(01,n) termwise multiplLication 
?3 Error USing ==> .* 


atrix dimensions must agree . 


1 = pinv(A1),A1'*(A1*A1')^-1,eye(Size(A1,2))/A1 55 AT[AAT] 一 
1= -0.1914 0.1399 s%right inverse of a2 X3matrix Al 


0.0617 0.0947 
0.2284 -0.0165 


*A1 1 %A1/A1 = 工 Implies the Validity of A1 1 as the right Inverse 
ns = 1.0000 0.0000 
0.0000 1.0000 


= Al ;5%5a3Xx2matrix 


1 = pinv(A5),(A5'*A5)^-1*A5',A5\eye(Size(A5,1)) % [AiA5]-1A5 


1= -0.1914 0.0617 0.2284 5%5Jeft inverse of a 3x2 matrix A5 
0.1399 0.0947 -0.0165 


_1*A5 555= 工 implies the validity of A5_1 as the left inverse 
ans = 1.0000 -0.0000 
-0.0000 1.0000 


_]1I1= (A1' *A1)^-1*A1” s%the Left inverse of matrix A1 with M < N? 
Warning: Matrix is close to Singular or badly scaled . 

Results may be inaccurate. RCOND = 9.804831e-018.， 
A1 11 = -0.2500 0.2500 

0.2500 0 

0.5000 0.5000 


(Q12) Does the left inverse of a matrix having rows fewer than columns exist 
(Al2) No. There is no N x M matrix that is premultiplied on the left of an M x N 


matrix with M < N to yield a nonsingular matrix, far from an identity matrix. 
In this context MAILAB should have rejected the above case on the ground 
that [4f4] is Singular and so its inverse does not exist. But, because the round- 
off errors make a very Small number appear to be a zero or make a real zero 
appear to be a very Small number (as will be mentioned in Remark 2.3), it ls 
not easy for MAILAB to tell a near-singularity from a real Singularity. That is 
why MAITLAB dares not to declare the Singularity case and instead issues just a 
warning message to remind you to check the validity of the result so that it will 
not be blamed for a delusion. Therefore, you mnust be alert for the condition 
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mentioned in item 2 of Remark 1.1, which says that, in order for the left inverse 
to exilst, the number of rows must not be less than the number of columns. 


>>A1_1Lix*Al %NOoO identity matrix，Since A1_11i isn't the left inverse 
ans = 1.2500 0.7500 -0.2500 
-0.2500 0.5000 0.7500 
1.5000 3.5000 2.5000 


>>det(A1'*A1) %A1 is not left-invertible for A1'*A1 is Singular 
ans = 0 


(cf) Let us be nice to MAITLAB as it is to us. From the standpoint of promoting mutual 
understanding between us and MAITLAB，we acknowledge that MAILAB tries to 
Show us apparently good results to Please us like always, Sometimes even pretending 
not to be obsessed by the demon of 'i-condition ”in order not to make us feel uneasy. 
How kind MAILAB is! But，we Should be always careful not to be Spoiled by its 
benevolence and not to accept the compnuting results every inch as it 18. In this case， 
even though the matrix [A1'*A1] is Singular and so not invertible, MAITLAB tried 
to invert it and that's all. MAILAB must have felt something abnormal as can be 
Seen from the ominous warning message prior to the computing result. Who would 
blame MAILAB for being So thoughtful and loyal to us? We might well be rather 
touched by its Sincerity and Smartness. 


In the above statements, we see the Slash(/)/backslash(\) operators. These oper- 
ators are used for righUleft division, rfespectively; B/Alils the Same as B*inv(A) and 
A\B ls the Same as inv(A)*B when Als invertible and the dimensions of A and B 
are compatible. Noting that B/Als equivalent to (A'\B' ) ,jet us take a close look 
at the function of the backslash() operator. 


>>X = ATNA1 5 an identity matrix? 
X = 1.0000 0 -0.8462 
0 1.0000 1.0769 
0 0 0 


(Q13) It seems that AIN\Al should have been an identity matrix, but it is not, contrary 
to our expectation. Why? 

(Al13) We should know more about the various functions of the backslash(A)，which 
can be Seen by typing help slash'” into the MAILAB Command window. Let 
Remark 1.2 answer this question in cooperation with the next case. 


>>A1*X - A1 %zero if X is the Solution to A1*X = A1? 
ans = 1.0e-015 * 0 0 0 
0 0 -0.4441 


Remark 1.2. The Function of Backslash (\) Operator. Overall, for the command 
A\B"”，MAILAB finds a_ solution to the equation A*X = B. Let us denote the 
Iow/column dimension of the matrix A by M and N. 


1. 开 matrix Ais Square and upperVlower-triangular in the Sense that all of 
its elements below/above the diagonal are zero, then MAILAB finds the 
solution by applying backward/forward substitution method (Section 2.2.1). 


20 


> 


3: 


4. 
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H matrix A is square，Symmetric (Hermitian)，and positive definite，then 
MAILAB finds the solution by using Cholesky factorization (9ection 2.4.2). 
H matrix A is square and has no Special feature, then MAILAB finds the 
Solution by using LU decomposition (Section 2.4.1). 

HL matrix A is rectangular, then MAILAB finds a solution by using QR 
factorization (Section 2.4.2). In case A jls rectangular and of full rank with 
Tank(A) = min(M,N), it will be the LS (least-squares) Solution [Eq. (2.1.10)] 
for M > N (overdetermined case) and one of the many solutions that is not 
always the same as the minimum-norm solution [Eq. (2.1.7)] for M < N 
(underdetermined case). But for the case when A is rectangular and has 
Iank deficiency，what MAILAB gives us may be useless. Therefore, you 
must pay attention to the warning message about rank deficiency，which 
might tell you not to count on the dead-end solution made by the backslash 
(\) operator. To fnd an alternative in the case of rank deficiency，you 
had better Tesort to Singular value decomposition (SVD). See Problem 2.8 
for details. 


FEor the moment， let us continue to try more operations on matrices. 


>>A1./A2 s%termwise right division 


ans 


ans 


ans 


>>Inv 


ans 


ans 


>>B“^2 
ansS 


= 1 | 1 
-2 Inf 2 
>>A1.\A2 %termwise left division 
= 1 1 1 
-0.5 0 0.5 
>>format rat，B^ 人 -1 %5represent the numbers (of 已 -1) in fractional form 
= 0 -1/3 
-1/2 -1/6 
(B) %inverse matrix，equivalent1y 
= 0 -1/3 
-1/2 -1/6 
>>B.^-1 s%termwise inversion(reciprocal of each element ) 
= 1 -1/2 
21/3 Inf 
%Ssquare of B，i.e.，B2 一 下 #* 孔 
= 7 -2 
-3 6 


>>B.^2 %termwise Square(Ssquare of each element) 


ans 


号 1(25 ) 4(D2) 
9(b) 0(22) 


>>2.^B %2 to the power of each number in B 


ans 


= 2 (20) 1/4(2252) 
1/8(222) 1 (222) 


>>A1.^A2 %element of A1 to the power of each element In A2 


ans 


= -1 (4 D200) 4(41(0,2)202) 27(4101,3)20 7) 
1116(4102,D200) 104102,2)200) 2(4102,3) 和 7) 


>>format Short，exp(B) %5elements of e3 with 4 digits below the dp 


ans 


= 2.7183(eb) 0.1353(ebp ) 
0.0498(ep) 1.0000(eo ) 


There are more useful MAILAB commands worthwhile to learn by heart. 
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了 R 了 emark 1.3. More Useful Commands for VectoYVMatrix Operations 


1. 


We can use the commands zeros()，ones()，and eye() to construct a 
matrix of specified Size or the Same Size as an existing matrix which has 
Only zeros, only ones, Or only ones/zeros on/off its diagonal. 


>>2Z = zeros(2,3) %or zeros(Size(A1)) yielding a 2 X 3 zero matrix 
Z = 0 0 0 
0 0 0 


>>E = Ones(Size(B)) %5or ones(3,2) yielding a 3 Xx2 one matrix 
E=1 1 


1 1 
1 1 

>>I = eye(2) %5yielding a 2 X 2 identity matrIxX 
I=1 0 
0 1 


. We can use the diag() command to make a column vector composed 


of the diagonal elements of a matrix or to make a diagonal matrix with 
on-diagonal elements taken from a vector given as the input argument. 


>>A1，diag(A1) #%scolumn vector consisting of diagonal elementsS 


A1= -1 2 3 
4 5 2 
ans = -1 
5 


.We can use the commands sum() /prod() to get the Sum/product of ele- 


ments in a vector or a matrix，columnwisely first (along the first non- 
Singleton dimenslon). 


>>Sal = Sum(a1) ssSum of all the elements in vector al 
sal=4%》al() 一 一 1 十 2 十 3=4 

>>SA1 = Sum(A1) ssum of all the elements in each column of matrix 41 
SA1=3 7 5 %s410) = 》Y 141I00,1) = [1 二 4 2 二 5 3 二 2] 


且 : 三 





>>SA1 = Sum(Sum(A1)) %Ssum of all elements in matrix 41 
SA1 = 15 %Ss41 = > _， YX_i4 =3+7+5= 15 
>>pal = prod(al) %product of all the elements in vector al 
pal=45%5[[a) = (-Dx2x3= 一 6 
>>pA1=product(A1) %product of all the elements in each column of matrix 4I1 
pA1 = -4 10 6 %p410) = TY _，4i0nm) = [-1x4 2x53 3x 72] 
>>PA1 = product(product(A1)) %sproduct of all the elements of matrix AI1 
PA1 = -240 spP41 = [II -4 = (- 和 0x10x6= -240 





. Wecanusethe commands max()/Vmin() tofindthe frst maximum/minimum 


number and its index in a vector or in a matrix given as the input argument. 


>>[aM,iM] = max(a2) 
aM = 5， = 2 %means that the max，element of vector a2 is a2(2) = 5 
>>[AM,IM] = max(A1) 
AM=4 5 3 
IM=2 2 1 
%means that the max， elements of each column of A1 are 
A1(2,1) = 4，A1(2,2) = 5，Al(1,3) = 3 
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>>[AMx,J] = max(AM) 
AMx =5，J= 2 
s%implies that the max. element of A1 is A1(IM(J) ,J) = A1(2,2) = 5 


53. We can use the commands rot90()/fliplIr()/flipud() torotate a matriX 
by an integer multiple of 90"” and to fip it left-rightVyup-down. 


>>A1，A3 = rot90(A1) ，A4 = rot90(A1,-2) 


AT1 = -1 2 3 
4 5 2 
A3= 3 2 %90” rotation 
2 5 
-1 4 
A4 = 2 5 4 %590"x(-2) rotation 
3 2 -1 
>>A5 = flipJr(A1) %5flip Jeft-right 
A5= 3 2 -1 
2 5 4 
>>A6 = flipud(A1) %5flip up-down 
A6 = 4 5 2 
-1 2 3 


6. We can use the reshape() command to change the row-column Size of a 
matrix with its elements preserved (columnwisely firstb). 


>>A7 = reshape(A1,3,2) 
A7 = -1 5 
4 3 
2 2 
>>A8 = reshape(A1,6,1)，A8 = A1(:) %makes Supercolumn Vector 
A8 = -1 
4 
2 
5 
3 
2 


1.1.8 Random Number Generators 


MAILAB has the built-in functions，rand()/randn()，to generate random 
numbers having uniform/mnormal (Gaussian) distributions，respectively ([ 开 -1]， 
Chapter 22). 


rand(M,N): generates an M x N matrix consisting of uniformly distributed 
random numbers 


randn(M,N): generates an M x N matrix consisting of normally distributed 
random numbers 
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1 PRanadom Nurmber Having Uniform Distrbuton 

The numbers in a matrix generated by the MAILAB function rand(M;N) have 

uniform probability distribution over the interval [0,1],， as described by U(0,1). 

The random number x generated by rand() has the probability density function 

1] Yx>0 
OO) 三 必 () 一 几 rr 一 DG) 一 | 0 V 0: the unit step function) 

(1.1.1) 

whose value is 1 over [0,1] and 0 elsewhere. The average of this standard uniform 

number X is 





1 











Oo 1 2 1 
111X 3 Xjx(C)dx = Xdx 一 一 | 一 一 (1.1.2) 
二 呈 0 歼 | 2 
and its variance Or deviation js 
允 1 1 1 人 es| 和 “ 林 
axX 一 /ec 一 mx) 六 Co)dx 本 (x 一 dx 一 了 (X 5 二 五 
(1.1.3) 


开 you want another random number y with uniform distribution U(a, D), trans- 
form the standard uniform number x as follows: 


y 一 (一 axz 十 4 (1.1.4) 


For practice,，we make a vector consisting of 1000 standard uniform numbers， 
transform it to make a vector of numbers with uniform distribution U( 一 1, 十 1])， 
and then draw the histograms showing the Shape of the distribution for the two 
uniform number vectors (Fig. 1.7a,b). 


>>uU_noise = rand(1000,1) s%a 1000x1 noise vector With U(0,1) 
>>Subplot(221)，hist(u_noise,20) s%histogram having 20 divisions 
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(c) Gaussian noise N(0, 1) (d) Gaussian noise N(0, 1/23) 
Figure 1.7 ”Distribution (histogram) of noise generated by the rand() /randn() command. 
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>>U_noise1 = 2*u_noise-1 %5a 1000x1 noise Vector With U(-1，,1) 
>>SubpJlot(222)，hist(u_noise1,20) %histogram 


2. Panaom Nurmber with Normal (Gaussianm) Distrbuton 
The numbers in a matrix generated by the MAILAB function randn(M,N) have 
normal (Gaussian) distribution with average 丸 一 0 and variance a?2 一 1，as 
described by NW(0,1). The random number x generated by rand() has the prob- 
ability density function 
1 2 
产 G) = 一 -< (1.1.5) 
V2 均 

H 开 you want another Gaussian number y with a general normal distribution 

NOz, oa2), transform the standard Gaussian number x as follows: 


yy 一 OCX 十 7 玉 (1.1.0) 


The probability density function of the new Gaussian number generated by this 
transformation is obtained by Substituting xY = (y 一 1m)/o into Eq. (1.1.3) and 
dividing the result by the Scale factor a (which can be seen im dx = dy/or) 
So that the integral of the density function over the whole interval (一 co, 十 co) 
amounts to 1 





广 (O) = eTOTOD 7120 (17) 


2 


For practice, we make a vector consisting of 1000 standard Gaussian numbers， 
transform it to make a vector of numbers having normal distribution NM(1,1/4)， 
with mean 17 一 1 and variance ca? = 1/4, and then draw the histograms for the 
two Gaussian number vectors (Fig. 1.7c,d). 


>>g_noise = randn(1000,1) %5a 1000x1 noise Vector With N(0,1) 
>>SubpJot(223)，hist(g_noise,20) %histogram having 20 divisions 
>>g_noise1 = g_noise/2+1 %a 1000x1 noise vector with N(1,1/14) 
>>SubpJlot(224)，hist(g_noise1,20) %histogram 


1.1.9 Flow Control 


1T. if-end anaq switch-case-end Statements 
An if-end block basically consists of an If statement, a Sequel part, and an emaq 
statement categorizing the block. An ir statement，having a condition usually 
based on the relationallogical operator (Jable 1.4), is used to control the program 
fow 一 that ls, to adjust the order in which statements are executed according to 
whether or not the condition is met, mostly depending on unpredictable situa- 
tions. The Sequel part conslsting of one or more Statements may contain eJSse OF 
elJseir statements, posSibly in a nested Structure containing another iT Statement 
inside 1it. 

The switch-case-end block might replace a multiple if-elseif-..-end 
statement in a neat manner. 
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Table 1.4 Relational Operators and Logical Operators 


人 
Operator 医 尖 Operator 


sm than ae seaermm | 


一 个 less than or equal to 医 下 greater than or equal to 一 过 


Let us see the following examples: 





EXa11D1e 7. 4 Si11D1e if-else-end BIocK 


%nm119_ 1: example of if-end block 
tt= 0 
ift>0 
Sgnt = 1; 
else 
sgnt = -1; 
end 





EXa11D1e 2. 4 Si11D1e if-elseif-end BocK 


%nm119 2: example of if-elseif-end block 
ift>0 
Sgnt = 
elSeif 研 < 0 
Sgnt -1 
end 





EXa11D1e 3 41 if-elseif-else-end BiocK 


%nm119 3: example of if-elseif-else-end block 
ift>0，sgnt=1 

elseif t<0，Ssgnt 

else Ssgnt = 0 

end 





EXa1iD1e4 41 if-elseif-elseif-,.-else-end BocK 


%nm119 4: example of if-elseif-elseif-else-end block 
point = 85; 
If point >= 90，grade = 'A' 

elseif point >= 80，grade 'B， 


elSeif point >= 70，grade = “C， 

elSeif point >= 60，grade = “'D， 

elSe grade = ' F， 
end 
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EXa11D1e 5. 4 Switch-case-end BiocK 


%nm119 5: examp1le of Switch-case-end block 

point = 85; 

Switch floor(point/10) %floor(x): integer less than or equal to X 
case 9，grade = 'A， 


case 8，grade = 'B' 

case 7，grade CC 

case 6，grade :四 : 

otherwise grade = 'F， 
end 





2. for index = 二 0:increment:i last-endLoop 

A for loop makes a block of statements executed Tepeatedly for a Specified 
number of times，with its loop index increasing from 1i .0 to a number not 
greater than 1i_ last by a _ specified step (increment) or by 1 让 not Specified. 
The loop iteration normally ends when the loop index reaches 1 last，but 
can be Stopped by a break statement inside the for loop. The for loop with a 
positive/negative increment will never be iterated if the last value (1_ last) of 
the index is Smallergreater than the starting value (1_0). 


下 Xda11D1e 06. 4 for ZLoop 


%nm119 6: example of for 10oop 
point = [76 85 91 65 87] ; 
forn= 1:1length(point) 
if point(n) >= 80， pfln,:) = 'pass'; 
elseif point(n) >= 0，pfl(n,:) = fail ' ; 
else  %if point(n)< 0 
pf(n,:) = ?3333 |; 
fprintf(' AnNXa Something wrong with the data??N\n ' ) ; 
break ; 
end 
end 
pf 





3. while Loop 
A while loop will be iterated as long as its predefined condition is Satisfied and 
a break Statement is not encountered inside the loop. 


ExXa11Dp1e 7 4 while ZLoo7p 


s%nm119 7: examp1le of while 10op 
1; 
while rr< 10 
= input('NnType radius (or nonpositive number to Stop): ) 


if rr<= 0，break，end %5isempty(r)| mm <= 0，break，end 
4/3x*xpixPrxrxr; 
fprintf( The volume of a Sphere With radius %53.1f = %58.2f\n' yyryv) 
end 
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EXa1iD1e 6. While ZLoops IO Fi2d 1pe MIi0I1UIWMMaXI1U1T Posifive NU1IDe18 


The following program “nml19 8.m”contains three while loops. In the first 
one, x = 1 continues to be divided by 2 until just before reaching zero, and 站 
will hopefully end up with the smallest positive number that can be represented 
in MAILAB. Im the second one, x = 1 continues to be multiplied by 2 until just 
before reaching inf (the infinity deftned in MAILAB), and seemingly it will get 
the largest positive number (x_max0) that can be represented in MAILAB. But， 
while this number reaches or may exceed inf 让 multiplied by 2 once more, it still 
is not the largest number in MAILAB (slightly less than inf) that we want to 
fnd. How about multiplying x_max0 by (2 一 1/2”)7 In the third while loop, the 
temporary variaple tmp starting with the initial value of 1 continues to be divided 
by 2 until just before x_max0ox* (2-tmp) reaches inf，and apparently it will end 
up with the largest positive number (x_max) that can be represented in MAILAB. 


%nm119 8: examp1le of while 1Loops 
1; kl1 = 0; 
While x/2 > 0 
X= X/2; kl1= kl + 1 
end 
k1，Xx_min = X; 
fprintf('x_min is %520.18e\n' ,Xx_min) 


1; k2 = 0; 
While 2*X < inf 
X = Xx2; Kk2 = Kk2+1; 
end 
k2，X_max0 = X; 


tmp = 1; k3 = 0; 
While Xx_maxOox*(2-tmp/2) < inf 
tmp = tmp/2; k3 = Kk3+1; 
end 
k3，Xx_max = X_maxox(2-tmp); 
fprintf('x_max 1IS %520.18e\n' ,，X_max) 


format 1Long e 
X_min,-Xx_miny,x_max，-X_max 
format hex 
X_min,-Xx_miny,x_max，-Xx_max 
format Short 





1.2 COMPUTER ERRORS VERSUS HUMAN MISTAKES 


Disgital Systems like calculators and computers hardly make a mistake, since they 
follow the programmed order faithfully. Nonetheless，we often encounter Some 
numerical errors in the computing results made by digital Systems, mostly coming 
from representing the numbers in finite bits, which is an intrinsic limitation of dig- 
ital world. If you let the compnuter compute Something without considering what 
is called the finite-word-length effect, you might come across a weird answer. In 
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that case, it ls not the computer, but yourself as the user or the programmer，who 
is to blame for the wrong result. In this context， we Should always be careful not 
to let the compnuter produce a farfetched output. In this section we will see how 
the compnuter represents and stores the numbers. Then we think about the cause 
and the propagation effect of computational error in order not to be deceived by 
unintentional mistakes of the computer and, it is hoped, to be able to take Some 
measures against them. 


1.2.1 IEEE 64-bit Floating-Point Number Representation 


MAILAB uses the IEEE 64-bit foating-point number System to represent all 
numbers. It has a word Structure consisting of the Sign bit，the exponent field， 
and the mantissa field as follows: 


03 62 52 51 0 


Each of these felds expresses %, 正 , and M of a number 三 in the way described 
below. 


。 Sign bit 
Sb 一 0 for positive numbers 
9， 11 fornegative numbers 


。 了 上 xponent field (bazb5lbso .……: bj5>z): adopting the excess 1023 code 


已 = Exp - 1023 = {0,1,... ,2 一 1= 2047} -1023 
= { 一 1023, 一 1022, . .. ,十 1023, 十 1024} 


一 1023 十 1 for | HP| < 2-1022(Exp = 00000000000) 
= 1 一 1022 ~ 十 1023 for 2-1022 < | 外 < 21024(normalized ranges) 
十 1024 for 士 co 


。 Mantissa field (b5lb5o ... blbo): 

In the un-normalized range where the numbers are So Small that they can be 
Tepresented only with the value of hidden bit 0, the number represented by the 
mantisSa 1S 

1MM = 0.bjlb50..… .bibo = [bjlb50.… .bibo] x 2 (1.2.1) 


You might think that the value of the hidden bit is added to the exponent, instead 
of to the mantissa. 

In the normalized range, the number represented by the mantissa together with 
the value of hidden blit bh 一 1 1s 


MM = 1.b5lb50… :blbo 一 工 十 [bjlb50.…:bibo]l x 2 
一 1 十 bjl x2-1 十 bjox2 一 十 .…: 十 blx2-51 十 bo x2-3 
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三 :本寺 2 村 2 双全 和 工人 二 文人 5 分 
={11+2- 1 二 2x2 .02 一 2)} 
={l1+A,1+2A,... ,1+02 -DA=2-- 和 AI(A=2-52) (1.2.2) 


The set of numbers 9, 天 ,and M，each represented by the Sign bit 9，the 
exponent field Exp and the mantissa field M, represents a number as a whole 


厂 = 士 M .22 (1.2.3) 


We classify the range of numbers depending on the value (已 ) of the exponent 
and denote it as 


RF 二 [22,225+0 with -1022 < 巨 < 十 1023 (1.2.4) 


In each range, the least unit 一 that ls, the value of LSB (least Significant bitb) OF 
the difference between two consecutive numbers represented by the mantissa of 
32 bits 一 18 

从 二 从 区 和 全 2 和 2 光 2 语 生 (1.2.5) 


Let us take a closer look at the bitwise representation of numbers belonging 
to each range. 


0. 0(zero) 


63 62 52 51 0 
S 1 000..， 0000 ,0000 0000..， 0000 0000 




















1，Un-normalized Range (with the value of hidden bit bh = 0) 
R_1023 一 [2-104, 2-1022) with Exp = 0, 天 = Exp 一 1023 十 1 = 一 1022 


S|1000..， 0000|00000000 ....， 00000001| (0 二 2-52) x25=(0 十 2-52) x 2-1022 























8$1000...， 0000T1T1111111 11 (0 上 +(252 -1)2-52) 一 (1 - 2-52)} x 2-1022 





Value of LSB: A_loz3 一 A_lo 一 2 
2. The Smallest Normallized Range (with the value of hidden bit bn = ]) 


R_ios = [2-1022, 2-1021) with Exp = 1, 巨 = Exp ~- 1023 = -1022 




















Si1000 ... 000110000 0000 .... 00000000| (1+0) x2E= (1+ 0) x 2-1022 
S1000 ..，_ 000110000 0000 “...， 00000001| (1+ 2-52) x 2-1022 
Si1000 ..， 0001111111111 ... 11111111| {(( + (252 - 1) 2-52) = (2 - 2-52)} x 2-1022 











Value of LSB: 入 _lio> 一 2 二 
3. Basic Normalized Range (with the value of hidden bit bh = ]) 
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Ro = [20,20) with Exp = 210 -1 = 1023, 巨 =Exp-1023=0 

















S1011 ..， 1111100000000 ...， 00000000| ((+0)x25=(1+0)x20=1 
SJ011 ... 111110000 0000 “...， 00000001| (1+2-52)x20 
S1011..， 11111111111T .1111111] {( +(252- 1) 2-52) = (2 - 2-52)}x20 











Value of LSB: Auo = 2 
4. The Largest Normalized Range (with the value of hidden bit bh = ]) 


Rio2 一 [21023, 21024) with Exp = 2 一 2 = 2046, 已 = Exp 一 1023 = 1023 





SI111 ...， 111010000 0000 ...， 00000000 (1+0)x2E=(1+0)x21023 
二 2-52) X21023 








SI111... 111010000 0000 ...， 00000001 














SI111... 1110111111111 ...， 11111111| {(1 + (252 - 1) 2-52) = (2 - 2-52)} x 21023 





Value of LSB: 和 A_lo>” 一 = 人 2 
$. 士 co(inf) Exp = 21 一 1 = 2047, 已 = Exp -- 1023 = 1024 (meaningless) 























01111 ... 111110000 0000 ....， 00000000 +eo 关 (1+0)x2E-=(1+0)x21024 
1T111 .1111100000000 .00000000| -一 *-(1+0)x25=-(1+0)x21024 
SI111 ... 111110000 0000 “....， 0000 0001| invalid (not used) 

SI111..， 111 1I11111111 .11111111| invalid (not used) 











From what has been mentioned earlier, we know that the minimum and max- 
imum positive numbers are, reSspectively， 


ma 一 (0 十 2 0) x2-1022 一 2-1004 一 4.9406564584124654 x 10 全 
一 (2 一 2 2) x2103 一 1.7976931348623157 x 10308 


This can be checked by running the program “nm119 8.m” in Section 1.1.9， 

Now, in order to gain Some idea about the arithmetic computational mecha- 
nlism，let'"s See how the addition of two numbers, 3 and 14, represented in the 
IEEE 64-bit foating number System, is performed. 


Digital-to-Binary Conversion 一 Normalization- 64-bit Representation 




































































3io = 11a =1.12 x2= 团 ,12 x 21024-1029 23 .1 [ai4 .0 
141o = 11102 = 1.112x 23 = 团 .112 x 21026-1023 四 到 了 
310 = 112> 23 ..， 1 
hidden bit 64-bit representation 11 
3io = 0 10241o 四 .10000..，...0 1410 = 11102 
+) 141o = 0 102610 由 11000..， -0 atiemment 
3io =0 10261o 回 .01100..，...0 
+)14io = 0 1026io 加 .11000..，...0 
carry bit = 1 
0 102610 10.00100..，...0 Binary-to-Decimal Conversion 
normalize GO 102710 。 团 00010 0 -1.0001ax 101027-10 的 =100012 = 站 x24+1xX20-1710 
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In the process of adding the two numbers, an allignment 1s made So that the 
two exponents in their 604-bit representations equal each other; and it will kick 
out the part Smaller by more than 52 bits，causing Some numerical error. For 
example, adding 2-23 to 230 does not make any difference, while adding 2 一 2 to 
230 does, as we can see by typing the following statements into the MATLAB 
Command window. 


>>X = 2 人 30j X+ 2 人-22 == X，X + 2 人-23 == X 
ans = 0(false) ans = 1(true) 


(cf) Each range has a different minimum unit (LSB value) described by Eq. (1.2.3). I 
implies that the numbers are uniformly distributed within each range. The closer the 
range is to 0, the denser the numbers in the range are. Such a number representation 
makes the absolute quantization error large/Small for large/small numbers, decreasing 
the possibility of large relative quantization error. 


1.2.2 Various Kinds of Computing Errors 


There are various kinds of errors that we encounter when using a computer for 
computation , 


。 7T7z1cationm 克 71o 六 Caused by adding up to a finite number of terms，while 
we Should add infinitely many termas to get the exact anSswer in theory. 


。Rorv1d-o 太 尼 r7or: Caused by Tepresenting/storing numeric data in finite bits. 


。 Oye1orwVUnderfow: Caused by too large or too Small numbers to be ITep- 
resented/stored properly in finite bits 一 more Specifically, the numbers hav- 
ing absolute values larger/smaller than the maximum ( 廊 ax)minimum( 太 in) 
number that can be represented in MATLAB. 


。 AMNegljigiple 4dditiom: Caused by adding two numbers of magnitudes differing 
by over 92 bits, as can be Seen in the last Section. 


李 


。 708 of Sig11jca1ce: Caused by a “bad subtraction,”which means a Sub- 
traction of a number from another one that is almost equal in value. 

。 尼 /Tor Masg1zjicatio1: Caused and magnified/propagated by multiplying/divi- 
ding a number containing a small error by a large/Ssmall number. 


。 Errors depending on the numerical algorithms，step Size, and So on. 


Although we cannot be free from these kinds of inevitable errors in Some degree， 
it ls not computers，but instead human beings，who must be responsible for 
the computing errors. While our computer may inslst on its innocence for an 
unintended lie, we programmers and users cannot escape from the responsibility 
of taking measures against the errors and would have to pay for being careless 
enough to be deceived by a machine. We Should, therefore, try to decrease the 
magnitudes of errors and to minimize their impact on the final results. In order 
to do So，we must know the Sources of computing errors and also grasp the 
computational properties of numerical algorithms. 
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For instance, consider the following two formulas: 


后 本 VX 
ie) = Vxz(Vr 十 1 一 V7)， 户 C) = 本 (1.2.6) 


These are theoretically equivalent，hence we expect them to give exactly the 
Same value. However, running the MAILAB program “nml122.m”to compute 
the values of the two formulas，we see a Surprising result that，as X increasesS， 
the step of 亡 (x) incoherently moves hither and thither，while 户 (x) approaches 
1/2 at a Steady pace. We might feel betrayed by the computer and have a doubft 
about its reliability. Why does such a fustering thing happen with 万 (xz)? It is 
because the number of Significant bits abruptly decreases when the Subtraction 
(Vxz 十 1 一 wx) is performed for large values of x，which is called “loss of 
Significance”. In order to take a close look at this phenomenon，let x 一 105. 
Then we have 


wx 十 1= 3.162277660168381 x 107 = 31622776.60168381 
wx = 3.162277660168379 x 10" = 31622776.60168379 


These two numbers have 92 Significant bits, or equivalently 16 significant digits 
(22 盖 1052x3/10 多 105) so that their significant digits range from 108 to 10-8. 
Accordingly, the least Significant digit of their Sum and difference is also the 
eighth digit after the decimal point (10-8). 


Vx 十 1 十 WVx = 63245553.20336761 
WVx 十 1 一 Wx = 0.00000001862645149230957 0.00000002 


Note that the number of significant digits of the difference decreased to 1 from 
10. Could you imagine that a Single Subtraction may kill most of the Significant 
digits?7 This ls the very “loss of Significance"，which ls often called“catastrophic 
cancellation . 


%nm122 


inline(' sqrt(x)*(Sqrt(Xx + 1) - Sdqrt(x)) ，X ); 
inline(' sqrt(x)./(Ssqrt(Xx + 1) + Sdqrt(x)) ，Xx' ) 
X=1; 
format Jong e 
for k = 1:15 


fprintf( At X=%15.0f，f1(x)=%20.18f，f2(x) = %20.18f' ，Xx,f1(x)，,f2(x)); 
X = 10*xX; 

end 

Sx1 = Sqrt(x+1); SX= Sqrt(x)}jd= Sxl - SXj S= SX1 + SXi 

fprintf(' sqrt(x+1) = %525.13f，Ssqrt(xX) = %525.13f  ，Sx1,Sx) 

fprintf(' diff = %25.23f，Sum = %25.23f ,dsS) | 
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>> nm122 

At x= 1，f1(x)=0.414213562373095150，f2(x)=0.414213562373095090 
At x= 10，f1(x)=0.488088481701514750，f2(x)=0.488088481701515480 
At x= 100，f1(x)=0.498756211208899460，f2(x)=0.498756211208902730 
At x= 1000，f1(x)=0.499875062461021870，f2(x)=0.499875062460964860 
At xX= 10000，f1(x)=0.499987500624854420，f2(x)=0.499987500624960890 
At X= 100000，f1(x)=0.499998750005928860，f2(x)=0.499998750006249940 
At X= 1000000，f1(x)=0.499999875046341910，f2(x)=0.499999875000062490 
At xX= 10000000，f1(x)=0.499999987401150920，f2(x)=0.499999987500000580 
At X= 100000000，f1(x)=0.500000005558831620，f2(x)=0.499999998749999950 
At X= 1000000000，f1(x)=0.500000077997506340，f2(x)=0.499999999874999990 
At xX= 10000000000，f1(x)=0.499999441672116520，f2(x)=0.499999999987500050 
At X= 100000000000，f1(x)=0.500004449631168080，f2(x)=0.499999999998750000 
At x= 1000000000000，f1(x)=0.500003807246685030，f2(x)=0.499999999999874990 
At x= “10000000000000，f1(x)=0.499194546973835970，f2(x)=0.499999999999987510 
At x= 100000000000000，f1(x)=0.502914190292358400，f2(x)=0.499999999999998720 


sqrt(x+1) = 31622776.6016838100000，sdqrt(x) = 31622776.6016837920000 
diff=0.00000001862645149230957，sum=63245553.20336760600000000000000 
1.2.3 Absolute/Relative Computing Errors 


The absolute/relative error of an approximate value x to the true value X of a 
Ieal-valued variable is defined as follows: 


sr 一 X(true value) 一 X(approximate value) (1.2.7) 
四 及 一 区 (2.8) 
Ox 一 码 二 一 六 .二 . 


Hthe least Signifticant digit (LSD) is the dth digit after the decimal point, then 
the magnitude of the absolute error ls not greater than half the value of LSD. 


lsz| = | 刁 一 zx 三 二 0 (1.2.9) 


Ithe number of significant digits is y, then the magnitude of the relative error 
is not greater than half the relative value of LSD over MSD (most Significant 
digit). 
lsr| _ IX-X 1 

ES 
| 芭 | | 区 | 2 





| 癌 必 10- (1.2.10) 


1.2.4 Error Propagation 


In this section we will see how the errors of two numbers, x and y, are propagated 
with the four arithmetic operations. Error propagation means that the errors in the 
input numbers of a process or an operation cause the errors in the output numbers. 

Let their absolute errors be sx and sy, Tespectively. Then the magnitudes of 
the absolute/relative errors in the Sum and difference are 


srty 一 (X 士 了 ) 一 人士 y)=(X 一 2 十 ( 了 一 访 王 sr 土 2y 
|sx+y| 和 |sz| 十 |sy| (1.2.11) 


|sx+y| | 各 lsx/ XI 十 sy 2 oz 十 11oy| 
IX 士 7| 一 IX 士 了 | IX 士 了 | 











lossy| = (1.2.12) 
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From this，we can See why the relative error 1 magnified to cause the“loss 
of significance”in the case of subtraction when the two numbers X and 了 are 
almost equal so that |X 一 了 | 之 0. 

The magnitudes of the absolute and relative errors in the multiplication/division 
are 


|sxy| 三 |X7 一 xy| 三 |X7 一 (和 二 so 十 sy)| 六 1Xsy 土 了 7er| 




















|sxy| 友 | 和 ley| 十 | 了 lex| (1.2.13) 
|sxsy| -sy| |sx| 
|oxy| 三 < -十 三 |ox| 十 |oy| (1.2.14) 
”XI II 
有 XX XXX 十 Srx [xs 一 了 ex| 
| 帮 庆 这 | 二 气 | 生 | 反 讨 一 
y 了 下 :83 了 
| 区 lsy| 十 | 了 ls 
区 同人 (1.2.15) 
lsxiy| sr lsy| 
losil= 一 < 二 + 一 |ol+loy| (1.2.16) 


[0 


This implies that, in the worst case, the relative error in multiplication/division 
Imay be as large as the Sum of the relative errors of the two numbers. 


1.2.5 Tips for Avoiding Large Errors 


In this section we will look over several tips to reduce the chance of large errors 
oOccurring in calculations. 

First, in order to decrease the magnitude of round-off errors and to lower the 
possibijlity of overflow/underflow errors, make the intermediate Tesult as close to 
] as possible in consecutive multiplication/division processes. According to this 
Iule,when computing xy/z，we program the formula as 


。 (XYy)/z when x and y in the multiplication are very different in magnitude， 
。X(y/z) when y and zx in the division are close in magnitude, and 
。 (YX/z)y when x and z in the division are close in magnitude. 


For instance, when computing y”"/e“ withx > 1andy > 1, we would program 
it as (y/e")” rather than as y"/e“, So that overflow/underflow can be avolided. You 
may verify this by running the following MAILAB program “nm125_1.m”. 


%nm125_1: 
Xx=36;jy= 1e16; 
forn= [-20 -19 19 201] 


fprintf( y^%52d/e^%52dx = %525.15e\n' nn yn/exp(nxXx) ); 
fprintf('(y/e^x)^%2d = 5%525.15e\n' ,n,(YyY/exp(x))^n) 
end 
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>>nm125_1 
y^-20/e^-20x = 0.000000000000000e+000 
(y/e^x)^-20 = 4.920700930263814e-008 
y^-19/e^-19x = 1.141367814854768e-007 
(y/e^x)^-19 = 1.141367814854769e-007 
8 
8 


y^19/e^19Xx = .761417546430845e+006 
(y/e^x)^19 = .761417546430843e+006 
y^20/e^20Xx = NaN 
(Y/e^Xx)^20 = 2.032230802424294e+007 


Second, in order to prevent“loss of Significance', it 1 Important to avoid a 
“bad subtraction ” (Section 1.2.2) 一 that is, a subtraction of a number from another 
number having almost equal value. Let us consider a Simple problem of finding 
the roots of a Second-order equation ax 十 px 十 c 一 0 by using the quadratic 
formula 

一 十 V 思 一 4ac 一 一 V 思 一 4ac 


ee (1.2.17) 


屎 1 
20 20 


Let |4ac| < 太 2. Then,， depending on the sign of D， a “bad subtraction”may be 
encountered when we try to find xl Or xz?，which ls the smaller one of the two 
Ioots. This implies that it ls safe from the“loss of Significance”to compnute the 
Ioot having the larsger absolute value first and then obtain the other root by using 
the relation (between the roots and the coefficients) X1X2 一 C/a. 

For another instance,， we consider the following two formulas,，which are ana- 
lytically the same, but numerically different: 


一 COSX Sin2x 


1 
万 (xz) 一 0 人 (1.2.18) 


X2(1 十 coSX) 


Itis safe to use 万 (xz) forx 之 立 Sincetheterm (1 十 cosx)in 户 (x)ilsa-“ bad sub- 
traction , while it js safe to use 户 (x) forx 久 0Sincetheterm (1 一 cosx)in 万 (xz) 
is a bad subtraction . Let's run the following MAILAB program“nm125_2.m” 
to confirm this. Below is the running result. This implies that we might use Some 
formulas to avold a “bad subtraction . 


%nm125_ 2: round-off error test 
f1 inline(' (1 - coSs(X))/XxX/X' ，Xx'); 
f2 inline(' Sin(Xx)*Sin(X)/X/X/(1 + cos(X)) 
for kK=0:1 
X= kx*pi;j tmp = 1; 
for kl1 = 1:8 
=X+tmp; 
0.8f， '，Xx1) 
18.12eji f2(x) = %518.12e' ，f1(x1),f2(x1) ); 


tmp = tmpx*0.1; X1 
fprintf(' At X 51 
fprintf( fl1(x) 5 
end 
end 
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>> nm125 2 
At x = 0.10000000，f1(x) = 4.995834721974e-001; f2(x) = 4.995834721974e-001 
At x = 0.01000000，f1(x) = 4.999958333474e-001; f2(x) = 4.999958333472e-001 
At x = 0.00100000，f1(x) = 4.999999583255e-001; f2(x) = 4.999999583333e-001 
At x = 0.00010000，f1(x) = 4.999999969613e-001; f2(x) = 4.999999995833e-001 
At x = 0.00001000，f1(x) = 5.000000413702e-001; f2(x) = 4.999999999958e-001 
At x = 0.00000100，f1(x) = 5.000444502912e-001; f2(x) = 5.000000000000e-001 
At x = 0.00000010，f1(x) = 4.996003610813e-001; f2(x) = 5.000000000000e-001 
At x = 0.00000001，f1(x) = 0.000000000000e+000; f2(x) = 5.000000000000e-001 
At x = 3.24159265，f1(x) = 1.898571371550e-001; f2(x) = 1.898571371550e-001 
At x = 3.15159265，f1(x) = 2.013534055392e-001; f2(x) = 2.013534055391e-001 
At x = 3.14259265，f1(x) = 2.025133720884e-001; f2(x) = 2.025133720914e-001 
At x = 3.14169265，f1(x) = 2.026294667803e-001; f2(x) = 2.026294678432e-001 
At x = 3.14160265，f1(x) = 2.026410772244e-001; f2(x) = 2.026410604538e-001 
At x = 3.14159365，f1(x) = 2.026422382785e-001; f2(x) = 2.026242248740e-001 
At x = 3.14159275，f1(x) = 2.026423543841e-001; f2(x) = 2.028044503269e-001 
At x = 3.14159266，f1(x) = 2.026423659946e-001; f2(x) = Inf 








It may be helpful for avoiding a “bad suptraction ”to use the Taylor series 
expansion ([W-l]) rather than using the exponential function directly for the 
computation of e*. For example, Suppose we want to find 


大 


户 (x) 三 一 atx 一 0 (1.2.19) 
区 


We canuse the Taylor series expansion up to justthe fourth-order ofe" aboutx =0 


“(0 G)(0 (0 
2 31! 41! 
__ 1 2 3 4 
三 耳 让 广 拉 本 7 牛 可 壮 而 
to approximate the above function (1.2.19) as 
人 1.2.20 
Po = 一 入 十 可 革 十 7 十 二 故人) (1.2.20) 


Noting that the true value of (1.2.9) is computed to be 1 by using the 志 HOPpital s 
rule ([W-1]),，we run the MAILAB program “nm125_ 3.m” to fnd which one of 
the two formulas 户 (x) and 广 (c) is better for fnding the value of the expression 
(1.2.9) at x 王 0. Would you compare them based on the running result Shown 
below? How can the approximate formula 访 (x) outrun the true one 户 (x) for 
the numerical purpose, though not usual? It is _ because the zero factors in the 
numeratordenominator of 户 (x) are canceled to set (xz) free from the terror of 
a “bad subtraction.” 
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%nm125 3: reduce the round-off error using Taylor series 
f3 = inline(' (exp(x)-1)/Xx' ，Xx')， 

f4 inline('((X/4+1)*Xx/3) + X/2+1' ，X' ) 

Xx=0; tmp = 1; 

for kl1 = 1:12 


tmp = tmpx0.1; x1 = x+ tmpi 

fprintf('At X = %14.12f，'，x1) 

fprintf('f3(xX) = %18.12e; f4(x) = %18.12e'，f3(x1),f4(x1) ) ; 
end 





>> nm125_3 
At x=0.100000000000，f3(x)=1.051709180756e+000; f4(Xx)=1.084166666667e+000 


尺 
口 


At x=0.010000000000，f3(x)=1.005016708417e+000; f4(X)=1.008341666667e+000 
At x=0.001000000000，f3(x)=1.000500166708e+000; f4(Xx)=1.000833416667e+000 
At x=0.000100000000，f3(x)=1.000050001667e+000; f4(Xx)=1.000083334167e+000 
At x=0.000010000000，f3(x)=1.000005000007e+000; f4(X)=1.000008333342e+000 
At x=0.000001000000，f3(x)=1.000000499962e+000; f4(Xx)=1.000000833333e+000 
At x=0.000000100000，f3(x)=1.000000049434e+000; f4(X)=1.000000083333e+000 
At x=0.000000010000，f3(x)=9.999999939225e-001; f4(Xx)=1.000000008333e+000 
At x=0.000000001000，f3(x)=1.000000082740e+000; f4(X)=1.000000000833e+000 
At x=0.000000000100，f3(x)=1.000000082740e+000; f4(Xx)=1.000000000083e+000 
At x=0.000000000010，f3(x)=1.000000082740e+000; f4(Xx)=1.000000000008e+000 
X=0 


At 


.000000000001，f3(x)=1.000088900582e+000; f4(x)=1.000000000001e+000 


1.3 TOWARD GOOD PROGRAM 


Amonsg the various criteria about the quality of a_ general program，the most 
important one ls how robust its performance is against the change of the problem 
properties and the initial values. A good program guldes the program users who 
don't know much about the program and at least give them a warning message 
without runtime error for their minor mistake. There are many other features 
that need to be considered, such as user friendliness, compactness and elegance， 
Ieadability, and So on. But, as far as the numerical methods are concerned, the 
accuracy of Solution，execution Speed (time efficiency), and memory utillzation 
(Space efficiency) are of utmost concern. Since Some tips to achieve the accuracy 
or at least to avolid large errors (including overflow/underflow) are given in the 
previous Section，we will look over the issues of execution Speed and memory 
utilization. 


1.3.1 Nested Computing for Computational Efficiency 


The execution Speed of a program for a numerical solution depends mostly on 
the number of function (Subroutine) calls and arithmetic operations performed in 
the program. Therefore, we like the algorithm requiring fewer function calls and 
arithmetic operations. For instance, Suppose we want to evaluate the value of a 
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polynomial 
D4(X) 一 CIX4 十 CDX3 十 C3X2 十 G4X 十 05 (1.3.1) 


It ls better to use the nested structure (as below) than to use the above form as 
it is. 
Pd4n(Cx) 一 (((GIX 十 02)X 十 03)xX 十 G4)X 十 G5 (1.3.2) 


Note that the numbers of multiplications needed in Eqs. (1.3.2) and (1.3.1) are 
4and (4 十 3 十 2 十 1=9), respectively. This point is illustrated by the Program 
“nm131_1.m”，wphere a polynomial | aixXi of degree N = 105 for a certain 
value of x ls computed by using the three methods 一 that jiSs，Eq. (1.3.1)，Eq. 
(1.3.2), and the MAILAB built-in function “polyval() . Interested readers could 
run this program to see that Eq. (1.3.2) 一 that is, the nested multiplication 一 1S 
the fastest, while “polyval() ”is the slowest because of some overhead time for 
being called, though it ls also fabricated in a nested Structure. 


snm131_ 1: nested multiplication vs，plain multiple multiplication 
N = 1000000+1; a= [1:N]; xx= 1 

tic 5% 1Initialize the 七 mer 

p= Sum(a.*x.^[N-1:-1:0]); %plain multiplication 

p，toc 55 measure the time passed from the time of executing “tic' 
tic，pn=a(1); 


for II = 2:N%nested multiplication 
pn = pn*sx+a(ir); 

end 

pn，toc 

tic，polyval(a,X)，toc 





Programming in a nested structure ls not only recommended for time-efficient 
computation，but also may be critical to the solution. For instance，conslder a 
problem of fnding the value 


天 天 
(天 ) 一 一 e 一 for 入 一 100 and 天 =155 (1.3.3) 


%nm131 2 _1: nested structure %nm131 2 2: not nested Structure 
lam = 100; K = 155; am 100; K = 155; 

p = exp(-Lam); ) 

S = 0 for k = 1:K 

1Lam^k/factorial(k) ; 

S + pi 


中 关口 1 咱 


for k = 1:K 

p=p*1lam/k;j S=S+p; S 
end end 
S Sx*exp(-1Lam) 





The above two programs are made for this computational purpose. Noting that 
this Sum of Poisson probability distribution ls close to 1 for such a large 开 ，we 
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can run them to fnd that one works fine，while the other gives a qulite wrong 
Iesult. Could you tell which one is better? 


1.3.2 Vector Operation Versus Loop lteration 


It is tme-efftcient to use vector operations rather than loop iterations to perform a 
Tepetitive job for an array of data. The following program “nm132_1.m”compares 
a vector operation versus a loop iteration in terms of the execution Speed. Could 
you tell which one ls faster? 


%nm132_1: Vector operation vS. Lo0op iteration 
N = 100000;i th = [0:N-1]/50000*pi; 


SS + Sin(th(I)); end 上 务 1oop iteration 


s%5 Vector operation 





As a more practical example, let us consider a problem of finding the DFT 
(discrete-time Fourier transform) ([W-3]) of a given Sequence X[7]. 


N 一 1 
X(O) = 》 xle- mo forQ = [一 100:100]r/100 (1.3.4) 


1 一 0 


The following program “nm132_2.m”compares a vector Operation VerSus a loop 
iteration for computing the DUFT in terms of the execution Speed. Could you tel 
Which one is faster? 


%nm132_2: nested structure 
1000; X = rand(1;N); 点 a random sequence x[n] for n 
:100]*pi/1100; 5 frequency range 


:Length (W) 
= 0; %for for 10op 
1:N，X1(k) = X1(k) + X(n)*xexp(-jx*W(k)*(n-1)); end 


:N  %for Vector 1oop 
2 +x(n)*exp(-j*Wx(n-1)); 


discrepancy = norm(X1-X2) %transpose for dimension compatibiIlity 
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1.3.3 lterative Routine Versus Nested Routine 


In this section we compare an iterative routine and a nested routine performing the 
Same job. Consider the following two Programs fctrl1(n)/fctrl2(n)，whose 
common objectives ls to get the factorial of a given nonnegative integer 上 . 


K! 一 KK 一 1 :2.1 (1.3.5) 


They differ in their structure. While fctrlL1() uses afor loop Structure, fctr121() 
uses the nested (Tecursive) calling Structure that aprogram uses itself as asSuproutine 
to perform a Sub-job. Compared with fctrlI1(),fctr1l2() ls easlier to program as 
well as to read, but is Subject to runtime error that ls caused by the exceSssive Use 
of Stack memory as the number of recursive calls increases with large n. Another 
disadvantage of fctrl2() is that it ls tme-ineffticient for the number of function 
calls，which increases with the input argument (n). In this case，a professional 
programmer would consider the standpoint of users to determine the programming 
style. Some algorithms like the adaptive integration (Section 3.8), however, may 
fit the nested structure perfectly. 


function m = fctrl1(n) 
m=1; 


forkK=2:n，m= mki end 





1.3.4 To Avoid Runtime Error 


A good program gulides the program users who don't know much about the 
program and at least gives them a warning message without runtime error for 
their minor mistake. If you don't know what runtime error 1$, YOU can experience 
one by taking the following Steps: 


1. Make and save the above routine fctrl1() in an Mr-file named “fctrl.m” 
in a directory listed in the MAILAB search path. 


2. Type fctrl(-1) into the MAILAB Command window. Then you will see 


>>fctrl(-1) 
anSs = 1 


This seemas to imply that (一 D)! = 1, which is not true. It ls caused by the mistake 
of the user who tries to fnd (一 DJ)! without knowing that it ls not defined. This 
kind of runtime error seems to be minor because it does not halt the ProcesSs. 
But it needs Special attention because it may not be easy to detect. 玉 you are a 
good programmer, you will insert Some error handling statements in the program 
fctrl() as below. Then，when Someone happens to execute fctrl(-1) in the 
Command window or through an M-file, the execution Stops and he will see the 
error message in the Command window as 


?3?? Error using ==> fctrlL 
The factorial of negative number ?? 
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function m = fctrln) 
ifn<0，error(' The factorial of negative number ?3? ' ) 


else m= 1; fork= 2:n，m= mxki end 
end 





This shows the error message (given as the input argument of the error() 
Ioutine) together with the name of the routine in which the accidental “error” 
happens，which ls helpful for the user to avoid the error. 

Most common runtime errors are caused by an “out of domain”index of array 
and the violation of matrix dimension compatibility, as lllustrated in Section 1.1.7. 
For example, conslider the gauss(A,B) routine in Section 2.2.2， whose job is to 
Solve a System of linear equations Ax = b for x. To appreciate the role of the fifth 
line handling the dimension compatibility error in the routine, remove the line 
(by putting the comment mark ss before the line in the Mr-file deftining gauss()) 
and type the following statements in the Command window: 


>>A = rand(3,3); B = rand(2,1); xX = gauss(A,B) 
?3? Index exceeds matrix dimensions . 
Error in ==> C:NXMATLAB6p5\nmaN\gauss .m 
on 1Line 10 ==> AB = [AU(1:NA,1:NA) B(1:NA,1:NB)]; 


Then MAILAB gives you an error message together with the Susplicious state- 
ment line and the routine name. But it ls hard to fgure out what causes the 
runtime error, and you may get nervous lest the routine Should have Some busg. 
Now, restore the fifth line in the routine and type the Same Statements in the 
Command window: 


>>X = gauss(A,B) 
?3? Error using ==> gauss 
AandBmust have compatible dimension 


This error message (provided by the programmer of the routine) helps you to 
realize that the source of the runtime error is the incompatible matrices/vectors A 
andB given as the input arguments to the gauss () routine. Very like this, a good 
program has a Scenario for possible user mistakes and fires the error routine for 
each abnormal condition to show the user the corresponding error message. 
Many users often give more/fewer input arguments than Supposed to be given 
to the MAITLAB functionsroutines and Sometimes give wIong types/formats of 
data to them. To experience this type of error，jlet us try using the MAILAB 
function sinc1(t,D) (Section 1.3.9) to plot the graph of a sinc function 


sin cf/D) = 人 with D=05 and 1!=[-2,2] (1.3.0) 
克 


With this purpose, type the following statements in the Command window. 
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| 上 上 上 上 


-2 一 1 0 1 2 -2 一 1 0 1 2 
(a) sinc1() with division-by-zero handling (b) sinc1() without division-by-zero handling 





Figure 1.8 The graphs of a sinc function defined by sinc1(). 


>>D = 0.5; bl1 = -2; b2=2;j tt= bl+[0:200]/200*(b2 - b1); 
>>plot(t,Ssincl(t,D))，axis([bl b2 -0.4 1.2]) 
>>hold on，plot(t,sinc1(t)，Kk: ) 


The two plotting commands coupled with sinc1(t,D) and sinc1(t) yield the 
two beautiful graphs，respectively，as depicted in Fig. 1.8a. It is Important to 
note that sinc1() doesn't bother us and works fine without the second input 
argument D. We oOwe the second line in the function sinc1() for the nice error- 
handling service: 


if nargin < 2，D = 1; end 


This line takes care of the case where the number of input arguments (nargin) ls 
less than 2, by assuming that the Second input argument js D = 1 by default. This 
programming technique is the key to making the MAILAB functions adaptive 
to different numbertype of input arguments，wphich is very useful for breathing 
the user-convenience into the MAILLAB functions. To appreciate its role，we 
Iemove the Second line from the Mr-file deftining sinc1() and then type the Same 
Statement in the Command window, trying to use sinc1() without the Second 
input argument， 


>>plot(t,Sinc1(t)，Kk: ) 
??? Input argument 'D' is undefined . 
Error in ==> C:NMATLAB6p5\nmaNxsinc1 .m 
On line4==>X= Sin(pix*t/D)./(pixt/D) ; 


This time we get a serious (Ted) error message with no graphic result. It is implied 
that the MAILAB function without the appropriate error-handling parts no longer 
allows the users default or carelessnesS. 

Now, consider the third line in sinc1(), which is another error-handling state- 
ment. 


t(find(t==0) )=eps; 
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or, equivalently 
for TIL = 1:length(t)，if t(i) == 0，t(Ii) = eps;i end，end 


This statement changes every Zero element in the t vector into eps (2.2204e- 
016). What is the real purpose of this statement? It ls actually to remove the 
possibility of division-by-zero in the next statement，which ls a mathematical 
expression having t in the denominator. 


Xx= Sin(pix*t/D)./(pix*t/D) ; 


To appreciate the role of the third line in sinc1(),， we Temove it from the Mr-file 
defining sinc1(), and type the following statement in the Command window. 


>>p1lot(t,Ssinc1(t,D)，r) 
Warning: Divide by zero . 
(Type "warning off MATLAB:divideByZzero" to Suppress this warning . ) 
In C:N\XMATLAB6p5\nmaN\sinc1l.m at line 4) 


This time we get just a warning (black) error message with a Similar graphic 
result as depicted in Fig. 1.8b. Does it imply that the third line is dispensable? 
No，because the graph has a (weird) hole at 上 = 0，about which most engi- 
neers/mathematicians would feel uncomfortable. That's why authors Strongly 
Iecommend you not to omit Such an error-handling part as the third line as 
well as the second line in the MAILAB function sinc1(). 


(cf) What is the value of sinc1(t,D) fort = 0inthis case? Aren't you curious? If so， 
let's go for it. 


>>Sinc1(0,D)，Sin(pi*0/D)/(pi*0/D)，0/0 
ans = NaN (Not-a-Number: undetermined) 


Last，consider of the fourth line in sinc1()，which is only one essential 
statement performing the main job. 


Xx= Sin(pix*t/D)./(pix*t/D) ; 


What is the .(dob before /(division operator) for? In reference to this，authors 
gave you a piece of advice that you had better put a .(dob just before the 
arithmetic operators *(multiplication)，/(division),， and“^(powern in the function 
definition So that the term-by-term (termwise) operation can be done any time 
(Section 1.1.6, (A5)). To appreciate the existence of the .(dot), we remove it from 
the M-file defining sinc1(), and type the following statements in the Command 
window. 


>>Clf，plot(t,sinc1(t,D))，Ssincl(t,D)，Sin(pix*t/D)/(pix*ty/D) 
ans = -0.0187 
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What do you see in the graphic window on the Screen? Surprise, a (horizontal]) 
Straight line running parallel with the 1-axis far from any sinc function graphl 
What is more Surprising, the value of sinc1(t,D) or sin(pixt/D)/(pixt/D) 
Shows up as a scalar. Authors hope that this accident will help you realize how 
important it ls for right term-by-term operations to put .(dot) before the arithmetic 
operators *, / and^ .By the way, aren't you curious about how MAILAB deals 
with a vector division without .(doby? Iso, let s try with the following statements: 


>>A = [1:10]; B = 2*A; A/B，A*B'*(BxB')^-1，Ax*pinv(B) 
ans = 0.5 


To understand this response of MAILAB，you can See 9ection 1.1.7 or Sec- 
tion 2.1.2. 

In this Section we looked at several Sources of runtime error, hoping that 革 
aroused the reader s attention to the danger of runtime error. 


1.3.5 _ Parameter Sharing via Global Variables 


When we discuss the runtime error that may be caused by user s default in passing 
some parameter as input argument to the corresponding function, you might feel 
that the parameter passing job is troublesome. Okay, it is understandable as a 
beginner in MAILAB. How about declaring the parameters as global So that 
they can be accessed/shared from anywhere in the MAILAB world as far as the 
declaration ls valld? I you want to, you can declare any varable(S) by inserting 
the following statement in both the main program and all the functions using 
the variables. 


global Gravity_Constant Dielectric_Constant 


%5plot_Ssinc 
Clear，Cclf 
global D 
1; b1 = -2; b2 = 2; 
= bl +[0:100]/100*x(b2 - b1); 
%passing the parameter(S) through arguments of the function 
Subplot(221)，plot(t，Sinc1(t,D) ) 
axis([bl b2 -0.4 1.2]) 


%5passing the parameter(S) through global variab1les 
Subplot(222)，plot(t，Ssinc2(t)) 
axis([bl b2 -0.4 1.2]) 


function X = Sinc1(t,D) function X = Sinc2(t) 

if nargin<2，D = 1; end global D 

t(find(t == 0)) = epsi t(find(t == 0)) = epsi 
Sin(pix*t/D)./(pix*t/D) ; X= Sin(pix*t/D)./(pix*t/D) ; 





Then, how convenient it would be,， Since you don't have to bother about pasS- 
ing the parameters. But, as you get proficient in programming and handle many 
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functions/routines that are involved with various Sets of parameters，you might 
fnd that the global variable is not always convenient，because of the follow- 
ing reasons. 


。Once a variable is declared as global, its value can be changed in any of the 
MAILLAB functions having declared it as global， without being noitced by 
other related functions. Therefore it is usual to declare only the constants as 
global and use long names (with all capital letters) as their names for easy 
identifcation. 


。 开 Some variables are declared as global and modified by several func- 
tions/routines, it ls not easy to see the relationship and the interaction among 
the related functions in terms of the global variable. In other words, the pro- 
gram readability gets worse as the number of global variables and related 
functions increases， 


FEor example, let us look over the above program “plot sinc.m”and the func- 
tion“sinc2()”. They both have a declaration of D as global; consequently， 
Sinc2() does not need the second input argument for getting the parameter 
D. I you run the program, you will see that the two plotting statements adopting 
sinc1() and sinc2() produce the Same graphic result as depicted in Fig. 1.8a. 


1.3.6 ”Parameter Passing Through Varargin 


In this section we see two kinds of routines that get a function name (String) 
with its parameters as its Input argument and play with the function. 

EFirst，let us look over the routine“ez_plot1()”， which gets a function name 
(ftn) with its parameters (p) and the lowervupper bounds (bounds = [bl1 b2]) 
as jlts first, third, and second input argument, respectively, and plots the graph of 
the given function over the interval set by the bounds. Since the given function 
may or may not have its parameter, the two cases are determined and processed 
by the number of input arguments (nargin) in the if-else-end block. 


s%plot_Ssincl 

Clear，ClLf 

D=1; bl= -2;j b2=2; 

t = bl+[0:100]/100*(b2 - b1); 

bounds = [b1 b2]; 
Subplot(223)，ez_plot1('sinc1' ,bounds,D) 
axis([bl1 b2 -0.4 1.2]) 
Subplot(224)，ez_plot('sinc1' ,bounds,D) 
axis([bl1 b2 -0.4 1.2]) 


function ez_plot1(ftn,bounds,p) function 

if nargin < 2，bounds = [-1 1]; end ez_plot(ftn,bounds,varargin) 

bl1 = bounds(1); b2 = bounds(2); if nargin < 2，bounds = [-1 1]; end 

七 = bl+[0:100]/100*(b2 - b1); bl1 = bounds(1); b2 = bounds(2); 

if nargin <= 2，X = feval(ftn,t); tt 上 =bl+ [0:100]/100*(b2 - bl); 
else X= feval(ftn,t,p); Xx = feval(ftn,t,varargin{f:}); 

end plot(t,X) 

plot(t,X) 





46 MATLAB USAGE AND COMPUTATIONAL ERRORS 


Now, let us see the routine“ez_plot()”， which does the Same plotting job 
as“ez_plot1()”. Note that it has a MAILAB keyword varargin (variable 
length argument listb) as its last input argument and passes lt into the MAILAB 
built-in function feval() as its last input argument. Since varargin can Tepre- 
Sent comma-separated multiple parameters including expresSion/strings, it paves 
the highway for passing the parameters in TelayS. As the number of parame- 
ters increaseS, it becomes much more convenient to use varargin for passing 
the parameters than to deal with the parameters one-by-one as in ez_plot1(). 
This technique will be widely used later in Chapter 4 (on nonlinear equations)， 
Chapter 3 (on numerical integration)，Chapter 6 (on ordinary differential equa- 
tions), and Chapter 7 (on optimization). 


(cf) Note that MAILAB has a built-in graphic function ezplot()，which is much more 
powerful and convenient to use than ez_plot(). You can type “help ezp1lot” to See 
its function and usage. 


1.3.7 Adaptive Input Argument List 


A MAILAB function/routine is Said to be “adaptive” to users in terms of input 
arguments it it accepts different numbertype of input arguments and makes a 
Ieasonable interpretation. For example， let us see the nonlinear equation Solver 
routine “newton() ”in 9Section 4.4. Its input argument list ls 


(f,df,Xxo,tol,kmax) 


where f，df，x0，tol and kmax denote the flename (String) of function (to 
be solved), the filename (String) of its derivative function, the initial guess (for 
solution), the error tolerance and the maximum number of iterations, respectively， 
Suppose the user, not knowing the derivative, tries to use the IToutine with just 
four input arguments as follows. 


>>newton(f,xo,tol,kmax ) 


At first, these four input arguments will be accepted as f,df,x0，and tol1， 
Iespectively. But，when the second line of the program body is executed, the 
Ioutine will notice Something wrong from that df is not any flename but a 
number and then interprets the input arguments as f,x0,to1l，and kmax to the 
idea of the user. This allows the user to use the routine in two waySs, depending 
on whether he ls going to Supply the routine with the derivative function Or not. 
This Scheme is conceptually quite Simijlar to function overloading of C++，but 
C++ Iequires us to have Several functions having the Same name，with different 
argument list. 


PROBLEMS 


1.1 Creating a Data File and Retrieving/Plotting Data Saved in a Data File 


(a) Using the MAILAB editor, make a program “nm1p01a”， which lets its 
user input data pairs of heights [ft] and weights [lb] of as many Persons 
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as he wants until he presses <Enter> and save the whole data in the 
form of an NM x2 matrix into an ASCII data file (***.dat) named by 
the user. 开 you have no idea how to compose Such a program，you 
can permnutate the statements in the box below to make your program. 
9tore the program in the fle named “nmlp01a.m”and run it to Save 
the following data into the data file named “hw.dat”: 


3.5102 
0.1185 
5.7170 
0.5195 
0.2191 


s%nm1p01a: input data pairs and save them into an ASCII data fil1e 
Clear 

k = 0; 

while 1 

end 

k=k+1; 

Xx(k,1) = hj 


hn = input('Enter height: ') 

x(k,2) = input('Enter weight: ') 

if isempty(h)，break;ji end 

cd(' cvmat1ab6p5\work ') %change current working directory 
filename = input('Enter filename(.dat):' ，S' ); 

filename = [filename ' .dat']; %string concatenation 
Save(filename，X'，' /ascii') 





(b) Make a MAILAB program“nm1p01b”，which reads (loads) the data 
fle“hw.dat” made in (a)，plots the data as in Fig. 1.1a in the upper- 
left region of the Screen divided into four regions like Fig. 1.3，and 
plots the data in the form of piecewise-linear (PWL) graph describing 
the relationship between the height and the weight in the upper-Tight 
Ieglion of the Screen. Let each data pair be denoted by the Symbol “十 ” 
on the graph. Also let the ranges of height and weight be [3$, 7] and 
[160,， 200], respectively. 开 you have no idea,， you can permnutate the 
statements in the below box. Additionally, run the program to check 让 
it works fine. 


s%nm1p01b: to read the data file and plot the data 

cd(' c:\matJlab6p5\work ' ) %5change current working directory 
weight = hw(I,2); 

1oad hw.dat 

clf，Subplot(221) 


plot(hw) 

Subp1lot(222 ) 

axis([5 7 160 2001]) 
plot(height,weight,height,weight，+ ' ) 
[height,I] = Sort(hw(: ,1)); 
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Text Printout of Alphanumeric Data 


Make a routine max_array(A), which uses the max() command to find one 
of the maximum elements of a matrix A given as jlts input argument and 
uses the fprintf() command to Print lt onto the Screen together with its 
row/column indices in the following format. 


'\n Max(A) is A(%52d,%52d) = %55.2fN\n row index;，col_index,maxA 


Additionally, try it to have the maximum element of an arbitrary matrix 
(generated by the following two consecutive commands) printed in this 
format onto the Screen. 


>>rand( state ,Sum(100*clock))，rand(3) 


Plotting the Mesh Graph of a Two-Dimensional Function 


Consider the MAILAB program“nm1p03a”，whose objective is to draw 

a cone. 

(a) The statement on the Sixth line Seems to be dispensable. Run the Pro- 
gram with and without this line and see what happens. 

(b) If you want to plot the function fcone(x,y) deftined in another M-file 
fcone.m ,how wa you modify this programy7 

(c) If you replace the fifth line by "Z = 1-abs(X)-abs(Y);， what differ- 
ence does it make?” 


s%nm1p03a: to plot a cone 

Clear，Clf 

Xx= -1:0.02:1;jyY = -1:0.02:1; 

[X,Y] = meshgrid(x,y); 
1-Sdqrt(X.^2+Y.^2) 


max(Z,zeros(Size(Z) ) ); 
mesh(X,Y,Z) 


function Z = fcone(Xx;,y) 
Z=1-sqrt(x.^2 + y.^2); 





Plotting The Mesh Graph of Stratigraphic Structure 


Consider the incomplete MAILAB program“nm1p04”，whose objective is 
to draw a stratigraphic structure of the area_ around Pennsylvania State 
University from the several perspective point of view. The data about 
the depth of the rock layer at 3 x 9 Sites are listed in Table P1.4. Sup- 
plement the incomplete parts of the program So that it Serves the pur- 
pose and run the program to answer the following questions. If you com- 
plete it properly and run it，MAILAB will show you the four similar 
graphs at the four corners of the Screen and be waiting for you to press 
any key. 
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(a) Atwhatvalue ofk does MAILAB show youthemesh/surface-type graphs 
that are the most similar to the first graphs? From this result, what do you 
guessarethe default values ofthe azimuth orhorizontal rotation angle and 
the vertical elevation angle (in degrees) of the perspective View point? 

(b) As the first input argument Az of the command view(Az,E1) decreases， 
in which direction does the perspective viewpoint revolve round the 
Z-aXlS，clockwise or counterclockwise (Seen from the above)? 

(c) Asthe secondinput argumentE1 ofthe command view(Az,E1) increases， 
does the perspective viewpoint move up or down along the z-axis? 

(d) What is the difference between the plotting commands mesh() and 
meshc()? 

(e) What ls the difference between the usages of the command view() 
with two input arguments Az,E1 and with a three-dimensional vector 
argument [xy;,z]? 


Table P1.4 The Depth of the Rock Layer 


X Coordinate 


y Coordinate 0.1 1.2 入 3 3.0 4.8 


0.5 410 390 380 420 450 
1.4 395 375 410 435 455 
222 365 405 430 455 470 
3:5 370 400 420 445 435 
4.0 385 395 410 395 410 


%nm1p04: to plot a stratigraphic structure 
Clear，Clf 

TOST2og ee 
Vy [0 5 -全 这 
乙 [ 汉 192990 总 ]5 


了 
了 


[X,Y] = meshgrid(x,y); 

Subplot(221)，mesh(X,Y,500 - ZI) 
Subplot(222)，Surf(X,Y,500 - ZI) 
Subp1lot(223) 
) 


，meshc(X,Y,500 - Z) 
Subp1lot(224)，meshz(X,Y,500 - Z) 
pause 
for k = 0:7 
AZz = -12.5*kj EL = 10*kj Azr = Azx*pi/1180; EL = EL*pi/1180; 
Subplot(221)，Vview(Az,E1) 
Subp1lot(222) ， 
k，Vview([sSsin(Azr),-cos(Azr),tan(ELIr)])，pause s%spause(1) 
end 





Plotting a Function over an Interval Containing Its Singular Point Noting 
that the tangent function 丰 (x) =tan(xz) is singular at x 一 范 /2, 3T/2， let us 
plot its graph over [0, 2 六 ] as follows. 
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(3) 


\D) 


(C) 


Define the domain vector x conslsting of sufficiently many intermediate 
point xi s along the x-axis and the corresponding vector y consisting 
of the function values at xi s and plot the vector y over the vector x. 
You may use the following statements. 


>>Xx = [0:0.01:2*pil]j y = tan(x); 
>>Subp1lot(221)，plot(x,y) 


Which one is the most simijlar to what you have got, among the graphs 
depicted in Fig. P1.37 JIS it far from your expectation? 

Expecting to get the better graph，we Scale it up along the y-axis by 
using the following command. 


>>axis([0 6.3 -10 10]) 


Which one is the most simijlar to what you have got among the graphs 
depicted in Fig. P1.3? JIs it closer to your expectation than what you 
got in (a)7 

Most probably，you must be nervous about the Straight lines at the 
Singular pointsx 王立/2andx 一 3r/2.The more disturbed you become 
by the lines that must not be there, the better you are at the numerical 
stuffs. As an alternative to avolid such a Singular happening,， you can 
try dividing the interval into three Sections excluding the two Singular 
points as follows. 
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Figure P1.5 Plotting the graph of fx = tanX. 


《d) 


(e) 
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>>x1 = [0:0.01:pi/2-0.01]; x2 = [pi/2+0.01:0.01:3*pi/2-0.01]; 
>>Xx3 = [3*pi/2+0.01:0.01:2x*pIl]; 
>>y1 = tan(x1); y2 = tan(x2); yY3 = tan(x3); 


>>Subplot(222)，plot(x1,y1,x2,y2,x3,yY3)，axis([0 6.3 -10 10]) 


Try adjusting the number of intermediate points within the plotting 
interval as follows. 


>>X1 [0:200]*pi/100; y1 = tan(x1) ; 
>>X2 [0:400]*pi/200; y2 = tan(x2) ; 
>>Subplot(223)，plot(x1,y1)，axis([0 6.3 -10 10]) 
>>Subp1lot(224)，plot(x2,y2)，axis([0 6.3 -10 10]) 


From the difference between the two graphs you got, you might have 
guessed that it would be helpful to increase the number of intermediate 
points. Do you Still have the Same ldea even after you adjust the range 
of the y-axis to [一 50, 十 530] by using the following command?” 


>>axis([0 6.3 -50 50]) 


How about trying the easy plotting command ezplot()2? Does it anSwWer 
yoOUI deslire7 


>>ezplot(' tan(x) ;0;,2*pIy) 


1.6 Plotting the Graph of a Sinc Function 

















The sinc function ls defitned as 
SinX 
J 太 (xc) 一 (P1.60.1) 
whose value atx 一 01s 
合生 Sinx _ (SinX) _ COSX 三 可 (P1.62) 
X-0 区 2 0 1 三 总 


We 
(3) 


(b) 





are going to plot the graph of this function over [一 4rr, 十 4 六 ]. 
Casually, you may try as follows. 


>>Xx = [-100:100]*pi/25; y = Sin(X)./X; 
>>plot(x,y)，axis([-15 15 -0.4 1.2]) 


In spite of the warning message about “division-by-zero ，you may 
Somehow get a_ graph. But ls there anything odd about the graph” 


How about trying with a different domain vector? 


>>X = [-4*pi:0.1:+4x*pil];j y = Sin(xX). /Xi 
>>plot(x;y)，axis([-15 15 -0.4 1.2]) 
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Surprisingly, MAILAB gives us the function values without any com- 
plaint and presents a nice graph of the sinc function. What is the 
difference between (a) and (b)? 


(cf) Actually, we would have no problem if we used the MATILAB built-in function 
Sinc(). 


Termwise (Element-by-Element) Operation in In-Line Functions 

(a) Let the function 方 (xz) be defined without one or both of the dot(.,) 
operators in Section 1.1.6. Could we still get the output vector consist- 
ing of the function values for the Several values in the input vector? 
You can type the following statements into the MAILAB command 
window and see the results. 


>>f1 


inline(' 1./(1+8*X^2) 
>>f1 


X ); 
inline(' 1/(1+8*Xx.^2) ，X' ) 


f1 
f1([0 1]) 


(b) Let the function 广 (x) be defined with both of the dot(.) operators as in 
Section 1.1.6. What would we get by typing the following statements 
into the MATLAB command window? 


>>f1 = inline(' 1./(1+8*x.^2) ，Xx'); fl1([0 1] ) 


In-Line Function and M-fle Function with the Integral Routine “quad ()” 


As will be seen in Section 9.8, one of the MAILAB built-in functions for 
computing the integral is “quad() , the usual usage of which is 


quad(f,aib,tol,trace,p1,p2,..) for 1 Jr D1DP2,，...)Gx 
(P1.8.1) 
Where 


f is the name of the integrand function (Mr-file name should be categorlized 
by ' )) 

aib are the lowerupper bound of the integration interval 

tol is the error tolerance (10-6 by default []) 

trace Set to 1(on)/0(ofb) (0 by default [ ]) for subintervals 

p1,p2,… are additional parameters to be passed directly to function 上 


Let's use this quad ( ) routine with an in-line function and an M-file function 
to obtain 


11 十 10 
1 (xc 一 X0) 太 (xc)CX (P1.8.2a) 


170 一 10 


and 
1 十 10 
1 G 一 X02 Cdx (P1.8.2b) 
一 10 
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where 


1 
x=1，jfo0= 一 -ee withm=lc=2  (P18.3) 
V2TG 


Below are an incomplete main program “nm1p08” and an M-file function 
defining the integrand of (P1.8.2a). Make another M-file defining the inte- 
grand of (P1.8.2b) and complete the main program to compute the two 
integrals (P1.8.2a) and (P1.8.2b) by using the in-line/M-file functions. 


function xfx = XxGaussian_pdf(x,m,Ssigma,Xx0) 
xfx = (X - X0).*xexp(-(x - m).^2/2/Sigma^2)/sdqrt(2*pi)/SsSigmay; 


%nm1p08: to try using quad() with in-1Line/M-file functions 

Clear 

m=1; sigma = 2 

int_xGausspdf = quad('` xGaussian pdf'm - 10,m+ 10,[],0,m sigma,1) 
Gpdf = ' exp(-(Xx-m).^2/2/Ssigma^2)/sdqrt(2*pi)/Sigma ; 

XxGpdf = inline(['(x - x0).*' Gpdf]，x' ，m'，Sigma'，Xx0' ); 
int_xGpdf = quad(xGpdf,m - 10,m+10,[],，0,m, sigma1) 





人 -Law Function Defined in an M-File 


The so-called -law function and -1L-law function used for non-uniform 
duantization ls defined as 


In(L 十 从 xl/zlmax) 


一 二 ma 一 一 一 Si P1.9 
> 三 8uCD) 三 |y| IE Sign(x) (P1.9a) 
本 (1 十 JPVpme 一 1 
x 一 87 0)== pa 一 sign0O) (P1.9b) 


Below are the -law function mulaw() defined in an M-file and a main 

program nm1p09，which performs the following jobs: 

。 Finds the values y of the -law function for x = [-1:0.01:1], plots the 
graph of y versus X， 


。Finds the values x0 of the /1-law function for y. 
。 Compnutes the discrepancy between x and x0. 


Complete the -1-law function mulaw_inv() and store it together with 
mulaw() and nm1p09 in the M-files named“mulaw_inv.m” “mulaw.m ， 
and “nmlp09.m”, respectively. Then run the main program nm1p09 to plot 
the graphs of the 人 -law function with 从 = 10, 30 and 255 and find the 


discrepancy between x and x0. 
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function [y, xmax] = mulaw(x;mu,ymax ) 
Xxmax = max(abs(x) ) ; 
y = ymaxx*Jog(1+muxabs(x/xmax))./1og(1+mu).*Ssign(X); 点 Eq.(P1.9a) 


function x = mulaw_ inv(y;mu,xmax) 


%nm1p09: to plot the mulaw Curve 


Clear，ClLf 

Xx= [-1:.005:1]; 

mu = [10 50 255]; 

for LIL = 1:3 
[y,xmax] = mulaw(x,mu(I) ,1); 
plot(x,y，b-'" ，Xx,x0,， rr-' )，hold on 
x0 = mulaw inv(y;mu(I) ,xmax) ; 
discrepancy = norm(Xx-XO) 

end 





1.10 Analog-to-Digital Converter (ADC) 


Below are two ADC routines adc1(a,b,c) and adc2(a,b;,c), which assign 
the corresponding digital value c(i) to each one of the analog data belong- 
ing to the quantization interval [b(i)，b(i+1)]. Let the boundary vector 
and the centrolid vector be, respectively， 


b=[-3-2-10123]; C=[-2.5 -1.5 -0.5 0.5 1.5 2.5]; 


(a) Make a program that uses two ADC routines to find the output d for 
the analog input data a = [-300:300]/1100 and plots d versus a to See 
the input-output relationship of the ADC, which is Supposed to be like 
Fig. P1.10a. 


function d = adc1(ab，c) 
%Analog-to-Digital Converter 
s#%5Input a = analog Signal，b(1:N + 1) = boundary Vector 
c(1:N)=centroid vector 
#%Output: d = digital SampJles 
Length(c) ; 
forn= 1:1ength(a) 
I=Tfind(a(n) < b(2:N)); 
If ~ISempty(I)，d(n) = c(I(1)); 
el1Se d(n) = CON) | 
end 
end 


function d=adc2(ab;c) 
Length(c) ; 
d(find(a < b(2))) = c(1); 
for ITL = 2:N-1 
index = find(b(I) <=a&a<= bl(i+l)); d(index) = c(I); 
end 
d(find(b(N) <= a)) = C(N) | 





PROBLEMS 55 
















































4 T T T T T T 
Output analog input 
2 上 ] | digital ] 
output 

0 上 二 四 | 
_2 上 了 ] 
疡 ， ，_input time 

一 4 -2 0 2 4 0 2 4 6 8 


(a) The input-output relationship of an ADC “”(b) The output of an ADC to a sinusoidal input 


Figure P1.10 The characteristic of an ADC (analog-to-digital converten). 


(b) Make a program that uses two ADC routines to find the output d for 
the analog input data a = 3*sin(t) with t = [0:200]/100*pi and 
plots a and d versus t to See how the analog input is converted into the 
digital output by the ADC. The graphic result is Supposed to be like 
Fig. P1.10b. 


1.11 Playing with Polynomials 


(a) Polynomial Evaluation: polyval() 
Write a MAILAB statement to compute 


poJ=xs-1 forx=1l (P1.11.1) 


(b) Polynomial Addition/Subtraction by Using Compatible Vector Addi- 
tion/Subtraction 
Write a MAILAB statement to add the following two polynomials: 


PIG) = 一刀 十 1 pa) 一 xz 一 2x2 十 1 (P1.11.2) 


(c) Polynomial Mujltiplication: conv ( ) 
WiiteaMAILAB statement to getthe following product of polynomials: 


po) 一 (人 TDo2z+Dx+DC 一 DT (P1.11.3) 


(d) Polynomial Division: deconv () 
Write a MAILAB statement to get the quotient and the remainder of 
the following polynomial division: 


PCD) 王妃 /(0 扩 一 1 (P1.11.4) 


(e) Routine for Differentiation/Integration of a Polynomial 
What you see in the below box jls the routine“poly_der(p)”，which 
gets a polynomial coefficient vector p (in the descending order) and 
outputs the coefficient vector pd of its derivative polynomial. Likewise， 
you can make a routine“poly_int(p)”， which outputs the _ coefficient 
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vector of the integral polynomial for a given polynomial coefficient 

Vector. 

(cf MAILAB has the built-in routines polyder()/polyint() for fnding the 
derivative/integral of a polynomial. 


function pd = poly_ der(p) 
%p: the Vector of polynomial coefficients in descending order 
N = length(p); 


ifN<= 1，pd=0ji 点 constant 
else 

for II=1:N- 1，pd(i) = pi)x*(N - I); end 
end 





(和 Roots of A Polynomial Equation: roots () 
WriteaMAILAB statement to get the roots of the following polynomilal 
equation 
poJ=x--1=0 (P1.11.5) 


You can check if the result is right, by using the MAILAB command 
poly()，which generates a polynomial having a given Set of roots. 
(g) Partial Fraction Expansion of a Ratio of Two Polynomials: residue()/ 
residuez() 
人 The MAILAB routine [r,p,k]l = residue(B,A) fnds the partial 
fraction expansion for a ratio of given polynomials B(s)/4(C) as 





M-1 M-2 ) 
8G) 人 ee 十 … 十 Dr 三 窜 洒 二 》 7(D) 
4(9) QI18 十 ay 十 … :十 GN 3 一 DO) 
(P1.11.6a) 
which is good for taking the inverse Laplace transform. Use this 
routine to find the partial fraction expansion for 
48 十 2 


十 692 十 11 二 6 一 十 we 二 
(P1.11.7a) 








X(y) 一 


( 廊 The MAILAB routine [r,p,k] = residuez(B,A) fnds the par- 
tial fraction expansion for a ratio of given polynomials B(z)/A4(z) 
as 


本 和 -CMV-D ; 

2(z) _ D1 二 十 世人 _ =kc-D+y 人 

4(C) al 十 az 十 .十 QZ-W-D 一 zz 一 PO) 

(P1.11.6b) 

which is good for taking the inverse z-transform. Use this routine 
to fnd the partial fraction expansion for 

4 十 2z-1 Z 这 2 

人 1 十 6z-1 十 11z 十 6z-3 xz 十 4 














区 
(P1.11.7b) 
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(h) Piecewise Polynomial: mkpp()/ppval() 
Suppose we have an MxN matrix P，the rowS of which denote 
让 (piecewise) polynomials of degree (NW-1) for diffterent Cnon- 
overlapping) intervals with (M + 1) boundary points bb = [b(1) 
，b(M + 1)]，where the polynomial coeffticients in each row are 
Supposed to be generated with the interval starting from x = 0. Then 
we can use the MATLAB command pp = mkpp(bb,P) to construct a 
Structure of piecewise polynomilals，which can be evaluated by using 
ppval(pp ). 
Figure P1.11(h) shows a set of piecewise polynomials {P1(x 十 3)， 
P2( 十 1),Pp3G 一 2)} for the intervals [一 3,， 一 HI],[ 一 1 2] and [2，4]， 
resSpectively，where 


Pio) 一 xj) 一 -一 1D2，and 1 一 xz 一 2 (P111.8) 


Make a MAILAB program which uses mkpp()/ppval() to plot thls 
graph. 

















-3 -2 一 1 0 1 2 3 X 4 


Figure P1.11(h) The graph of piecewise polynomial functions. 


(cf You can type help mkpp” to see a couple of examples Showing the usage 
of mkpp. 


1.12 Routine for Matrix Multiplication 


Assuming that MAILAB cannot perform direct multiplication on Vectors/ 
matrices, Supplement the following incomplete routine“mu1ltip1LIy_matrix 
(A,B)”so that it can multiply two matrices glven as its input arguments only 
it their dimensions are compatible，bput displays an error message 计 their 
dimensions are not compatible. Try it to get the product of two arbitrary 
3 x 3 matrices generated by the command rand(3) and compare the result 
with that obtained by using the direct multiplicative operator *. Note that 
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the matrix multipllication can be described as 


天 - 
COWm ,7) 一 >》 4(0n， 7 (P1.12.1) 
天 一 


function C = multiply_matrix(A,B) 
[M,K] = Size(A); [K1,N] = Size(B) 
if K1 -~= K 
error( The # of columns of Ais not equal to the # of rows of B') 
el1Sse 
form= 1: 
forn=1: 


Cmin) = Am,1)*B(1,n); 
for kK = 2: 
COmin) = Cm;n) + Amk)*B(k,n); 
end 
end 
end 
end 





Function for Finding Vector Norm 


Assuming that MAILAB does not have the norm() command finding us the 
norm of a_ given vectormatrix, make a routine norm_vector(v,p)，which 
computes the norm of a given vector as 


lvls = ” (P1.13.1) 


N 
》 |uml 
ET 





for any positive integer p, fnds the maximum absolute value of the elements 
forp = inf and computes the norm as 让 p = 2, even it the Second input 
argument p ls not given. I you have no idea, permutate the statements in the 
below box and Save it in the file named “norm_vector.m" .Additionally, try 
it to get the norm with p = 1,2,co (inf) and of an arbitrary vector generated 
by the command rand(2,1). Compare the result with that obtained by using 
the norm() command. 


function nyv 


= norm_vector(vp) 
if nargin < 2 
( 
( 


p = 2; end 
.pp) (1/p) ; 
) ; 


了 
nv = Sum(abs(Vv 
V 


) 
) 


nv = max(abs 
ifp>0&p-= inf 
elseif p == Inf 
end 
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1.14 Backslash (\) Operator 


Let's play with the backslash() operator， 

(a) Use the backslash() command, the minimum-norm solution (2.1.7) and 
the pinv() command to Solve the following equations, fnd the residual 
error | 4iX 一 bs and the rank of the coefficient matrix 4i， and fll in 
Table P1.14 with the results. 


屎 1 

5 1 2 3 0 

(iD hx=|4 5 | X2 -| = (P1.14.1) 
3 
X1 

(这 4x=| 习 X2 -| | = (P1.14.2) 
.3 
X1 

(iii) 4x=| 。 X2 -| 虽 |=w (P1.14.3) 
X3 


Table P1.14 _ Results of Operations with backslash (\) Operator and pinv( ) Command 


4.4409e-15 
(1.9860e-15) 





(cf) When the mismatching error | 4iX 一 bils obtained from MATILAB 5.x/6.x version are slightly different, the 
former one is in the parentheses (). 
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(b) 


(cb) 


(C) 


Use the backslash (A) command, the LS (least-squares) solution (2.1.10) 
and the pinv() command to solve the following equations and find the 
Tesidual error | 4iX 一 bs and the rank of the _ coefficient matrix 4,， 
and fll in Table P1.14 with the results. 


1 2 2 

D 4x=| 2 3 国 6 | = bs (P1.14.4) 
3 4 E 7 
1 2 1 

(GD 4x=|2 4 的 一 bs (P1.14.5) 
3 6 8 
1 2 3 

Gib 4ox=|2 4 的 6|=bs (P1.14.6) 
3 6 2 9 


H 开 some or all ofthe rows ofthe coefticient matrix 4 in a set of linear equations 
can be expressed as a linear combination of other row(S), the corresponding 
equations are dependent, which can be revealed by the rank deficiency, that is， 
Iank(4) < min(AM, NW) where M and N are the row dimension and the column 
dimension, respectively. Isome equations are dependent they may have either 
jnconsistency (no exact Solution) or redundancy (infinitely many Solutions)， 
which can be distinguished by checking ii augmenting the RHS vector b to the 
coefficient matrix 4 increases the rank or not 一 that is, rank([4 b]) > Tank(4) 
or not [M-2|]. 

Based on the results obtained in (a) and (b) and listed in Table P1.14， 
answer the following questions，. 


(iD Based on the results obtained in (a)()，which one yielded the 
non-minimum-norm solution among the three methods，that is， 
the backslash() operator, the minimum-norm solution (2.1.7) and 
the pinv() command? Note that the minimum-norm solution 
means the solution whose norm (xl is the minimum over the 
many Solutions. 

(ii Based on the results obtained in (al)，which one is most reliable 

as a means of fnding the minimum-norm solution among the 

three methods?7 

(ii) Based on the results obtained in (b), choose two reliable methods 

as a means of finding the LS (least-squares) solution among the 

three methods，that is，the backslash (\) operator，the LS Solu- 


tion (2.1.10) and the pinv() command. Note that the LS solution 
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Imeans the solution for which the residual error (|4x 一 blD is the 
Iminimum over the many Solutions. 


1.1S Operations on Vectors 


(a) Find the mathematical expression for the computation to be done by 
the following MAILAB statements. 


>>n = 0:100; S = Sum(2.^-n) 


(b) Write a MAILAB statement that performs the following computation. 


10000 1 本 

人 (7 十 地 本 

(c) Write a MAILAB statement which uses the commands prod() and 
sum() to compute the product of the sums of each row of a 3x3 
random matrix. 

(d) How does the following MAILAB routine“repetition(x;M,m)”con- 
vert a given IOowW vector sequence x to make a new sequence y ? 


function y = repetition(x,M;m) 
If m == 

MNX = ones(M,1)*xj y = MNX(:) ; 
el1se 

NX = length(x); N = ceil(NX/m); 
X= [xzeros(1;Nxm - NX)]; 

MNX = ones(M,1)*x; 


YE= 沁 [ 同 
for n 1:N 
tmp MNXx(:,， (nn - 1)xm+ [1:m]).; 
yY= [ytmp(:)，]; 
end 
end 





(el) Make a MAITLAB routine“zero_insertion(x;M;m)”，which inserts 
m Zeros just after every Mth element of a_ given row vector Sequence 
x to make a new Sequence. WIite a MAILAB statement to apply the 
Ioutine for inserting two zeros just after every third element of x 一 
[1 3 7 2 4 9]toget 


y=[1370024.9 0 0|] 


( How does the following MAILAB routine“zeroing(x,M;,m)”convert 
a given IOw vector sequence x to make a new sequence y?7 
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function yY = zeroing(x;,M,my) 

%Zzero out every (KM - m)th element 
if nargin < 3,m= 0; end 

if M<=0，M= 1; end 


m=mod(m;M)， 
Nx = Length(x); N = floor(NX/M) ; 
yY=Xxiy(M*II:N] - m) = 0 





(5g) Make a MAILAB routine“sampling(x;M,m)”，which Samples every 
(kM - m)th element of a given TOw vector Sequence x to make a new 
sequence. WIite a MAILAB statement to apply the routine for sampling 
evelry (3K 一 2)th element ofx=[1 3 7 2 4 9]toget 


y=[12] 


(hb) Make a MAILAB routine“rotation_F(x,M)”， which rotates a given 
IOW Vector Sequence x Tight by M Samples, Say, making rotate_r([i 
2345];3) = [345121]. 


Distribution of a Random Variable: Histogram 


Make a routine randu(N,a,b),， which uses the MATLAB function rand () 
to generate an N-dimensional random vector having the uniform distribution 
over [ab] and depicts the graph for the distribution of the elements of 
the generated vector in the form of histogram divided into 20 sections as 
Fig.1.7. Then, see what you get by typing the following statement into the 
MAILAB command window， 


>>randu(1000,，-2,2) 


What is the height of the histogram on the average? 
Number Representation 


In Section 1.2.1，we looked over how a number is represented in 64 bits. 
For example, the IEEE 64-bit foating-point number System represents the 
number 3(21 < 3 < 22) belonging to the range RiI = [21, 22) with 了 一 1 as 





3 


100 0000 0000|10000000 0000 0000 0000 0000 0000 0000 

















4 0 0 8 0 9 0 0 0 0 0 


where the exponent and the mantissa are 
Exp = 下 十 1023 = 1 十 1023 = 1024 = 210 = 100 0000 0000 
M=(3x2- 一 1Dx222=251 
三 1000 0000 0000 .... 0000 0000 0000 0000 0000 
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This can be confirmed by typing the following statement into MAILAB 
command window、. 


>>fprintf('3 = %bx\n' ,3) Or >>format hex，3，format Short 
which will print out onto the Screen 
0000000000000840 4008000000000000 


Noting that more significant byte (8[bits] = 2[hexadecimal digits]) of a 
number is stored in the memory of higher address number in the INTEL 
System，we can TIeverse the order of the bytes in this number to see the 
number having the mostyleast Significant byte on the leftright Slide as we 
can see in the daily life. 


00 00 00 00 00 00 08 40 一 40 08 00 00 00 00 00 00 


This ls exactly the hexadecimal representation of the number 3 as we 
expected. You can find the IEEE 64-bit foating-point number represen- 
tation of the number 14 and use the command fprintf() or format hex to 
check if the result is right. 


<procedure of adding 2-1 to 23> <procedure of subtracting 2-1 from 23> 





























































































































































































































四 .0000x23 四 .ooooGx 23 ， 回 .0o000x23 四 .0000x23 2s 1 
寺 alignment | 1 alignment 人 complement 
+ 山 .0000 x 2” 一 +gl.000 1- 山 .0000 x 2 ”一 -0 .00010:x 2” 一 一 十 . 
，/ 目 .0001x23 ， normalization /0 .1111 洋 x 23 
truncation of guard bit 站 ES 28 truncation of guard bit 4 证 和 和 22 
=(1+24) x23 =(1+1-2-3) x22 
right result right result 
<procedure of adding 2-2 to 23> <procedure of subtracting 2 -2 from 23> 
和 .0000x23 1.0000x23 ， 罩 .0000x23 1 23 2s 
9 alignment 1 2 alignment 3 complement 
+ 山 .0000 x 2 一 +O.000| 1 - 山 .0000 x 2 一 一 0 2” -一 一 一 +| 川 . 
truncation of guard bit( 媳 .00001x2 ， em ep 0.11114x2 
局 汪 .0000 久 23 ， runcatlon of guar 必 局 二 X 22 
=(1+0) x23 ， =(1+1-2 人 xx22 
no difference right result 
(cf [] :hidden bit :guard bit 











Figure P1.18 Procedure of addition/subtraction with four mantissa bits. 


Resolution of Number Representation and Quantization Error 


In Section 1.2.1，we have seen that adding 2- 2 to 230 makes Some dif- 
ference，while adding 2-23 to 230 makes no difference due to the bit shift 
by over 52 bits for alignment before addition. How about subtracting 2- 亿 
from 230? In contrast with the addition of 2-23 to 230, it makes a differ- 
ence as you can See by typing the following statement into the MAILAB 
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command window. 
>>X = 2^30j X+ 2 人 -23== X， X -2 人 -23 == X 


which will give you the logical answer 1 (true) and 0 (false). Justify this 
result based on the difference of resolution of two ranges [230, 231) and [222， 
230) to which the true values of computational results (230 十 2-23) and (230 一 
2-3) belong,respectively. Note from Eq. (1.2.5)thatthe resolutions 一 thatis， 
the maximum quantization errors 一 are Ap 一 22- 了 2 一 2-22130 一 2 一 2 and 
2 一 522 一 2 3,respectively.For details,referto Fig. P1.18, which illustrates 
the procedure of addition/subtraction with four mantissa bits, one hidden bit， 
and one guard bit， 





1.19 Resolution of Number Representation and Quantization Error 


(a) What is the result of typing the following statements into the MAILAB 
command window? 


>>7/100*x100 - 7 


How do you compare the absolute value of this answer with the reso- 
lution A of the range to which 7 belongs? 

(b) Find how many numbers are Susceptible to this kind of quantization 
error caused by division/multiplication by 100，among the numbers 
from 1 to 31. 

(c) What will be the result of running the following program2 Why?7 


s%nm1p19: Quantization Error 
x = 2-2^-50; 
forn= 1:2?3 


X = X+2^-52j; fprintf( %20.18ENn' ,X) 
end 





1.20 Avoiding Large Errors 人 Overflow/Underflow 
(a) Forx = 9.8201 and y = 10.219?, evaluate the following two expressions 
that are mathematically equlivalent and tell wphich ls better in terms of 
the power of reslisting the overfow. 


DZz=Vz2 十 入 (P1.20.1a) 
(四 zz=yVC/y)2 十 1 (P1.20.1b) 


Also forx = 9.8-201 and y = 10.2-19, evaluate the above two expres- 
Sions and tell which is better in terms of the power of resisting the 
underfow， 

(b) Witha =c=1andfor 100 values of pp over the interval [1074, 1085] 
generated by the MAILAB command “1ogspace(7.4,8.5,100) ， 


(C) 


〈d) 


(e) 
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evaluate the following two formulas (for the roots of a quadratic 
equation) that are mathematically equivalent and plot the values of the 
Second root of each pair. Noting that the true values are not available 
and so the shape of solution graph ls only one Practical baslis on which 
we can assess the quality of numerical Solutions, tell which is better in 
terms of reslsting the loss of Significance. 


g) 已 2 一 于 (- 干 sign(D)VZ 瑟 二 | (P1.20.2a) 
人 攻 三 二 (- 一 sign(D)VZ 二 40o,z 一 二 (P1.20.2b) 
此 


For 100 values of xz over the interval [1004, 1016], evaluate the follow- 
jing two expressions that are mathematically equlivalent, plot them, and 
based on the graphs, tell which is better in terms of reslsting the loss 
of Significance. 


人 y=VvVZzTIT_-1 (P1.20.3a 

介 2 (P1.20.3b) 

JI == 汪 网 
2x2 十 1 十 1 


For 100 values of x over the interval [10-?, 10-74], evaluate the fol- 
lowing two expressions that are mathematically equivalent, plot them， 
and based on the graphs, tell which is better in terms of reslisting the 
loss of significance. 


人 本 双语 (P1.20.4a) 
1 

这 三 P1.20.4b 

SS 人 ) 


On purpose to find the value of (300125/1251)e-300, type the following 
statement into the MAILAB command window. 


>>300^125/prod([1:125])*exp(-300) 


What is the result? Is it of any help to change the order of multipli- 
cation/divislion? As an alternative, make a Toutine which evaluates the 
expresSion 


入 K 
P(K) 一 人 for 入 一 300 and an integer 大 (P1.20.3) 


in a recursive way, Say, like P(E 十 1 =P(O* 和 KK and then use the 
routine to find the value of (300125/11251)e-300. 
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(人 Make a routine which computes the sum 


天 天 
入 
S(K) 一 for 和 = 100 and an integer 玉  (P1.20.0) 


-一 e 
有 ! 
=-0 


and then, use the routine to find the value of (155). 


1.21 Recursive Routines for Efficient Computation 


(a) The Hermite Polynomial [下 -]] 
Consider the Hermite polynomial defined as 


CN 
丽 =1 Ho0=(Dyer 一 e (P1.21.1) 
CQXN 
(人 Show that the derivative of this polynomial function can be wTit- 
ten as 
N+1 


一 2 





0 (一 DX2xer 和 ez 十 (一 Dwer e 
ee GXN 


CXN+1 
一 2x7 厅 N(X) 一 已 w+1(CY) (P1.21.2) 


and So the (N 十 l)th-degree Hermite polynomial can be obtained 
recursively from the Nth-degree Hermite polynomial as 


已 w+HIG) 一 2x7N(G) 一 五 VCO) (P1.21.3) 


(说 Make a MAILAB routine “Hermitp(N)”which uses Eq. (P1.21.3) 
to generate the Nth-degree Hermite polynomial 囊 w (xz)， 
(b) The Bessel Function of the First Kind [天 -]] 
Consider the Bessel function of the first kind of order K defined as 


(CD) = coOS(K6 一 6Ssin6)qd6 (P1.21.4a) 
0 


pb 忌 (人 一 Drp2r 

三 守 | 过 一 一 一 一 一 一 三 (一 1 六 P1.21.4b 

( CE (一 De(0B) ( ) 

(iD Define the integrand of (P1.21.4a) in the name of "Bessel_inte- 
grand(xibeta,k)” and store it in an M-file named “Bessel 
integrand .m”. 

( 廊 Complete the following Toutine “Jkb(K,beta)”，which uses 
(P1.21.4b) in a recursive way to compute (8B) of order K 一 
1:K for given K and 8 (beta). 

(iii) Run the following program nm1p21b which uses Eqs. (P1.21.4a) 
and (P1.21.4b) to get Ji5(B) for 8 = 0:0.03:13. What ls the norm 
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of the difference between the two results? How do you compare 
the running times of the two methods? 


(cf) Note that Jkb(K,beta) computes .人 (B) of order 上 = 1:K, while the inte- 
gration does for only 大 王 K. 


function [J,JJ] = Jkb(K,beta) s%the 1st kind of kth-order Bessel ftn 
tmpk = ones(Size(beta) ) ; 
for kK 0 :K 
tmp tmpk; JJ(k+ 1,:) = tmp; 
for = 1:100 
?3333333333?33333?3333?2? |; 
JJ(k+ 1，:) =JJ(k+ 1，:)+ tmnp; 
if norm(tmp)<.001，break;i end 
end 
tmpk = tmpk.*beta/2/(k + 1); 


JJ(K+1，:) 


%nm1p21b: Bessel_ftn 
Clear，Clf 

beta = 0:.05:15; K = 15; 
tiC 


for TIL = 1:1length(beta) %Integration 

J151(i) = quad('Bessel_integrand' ,0;,pi,[],0,beta(I),K)/pii 
end 
tocC 
tic，Jd152 = Jkb(K,beta); toc s%5Recursive Computation 
discrepancy = norm(J151-J152) 





1.22 Find the four routines in Chapter 9 and 7, which are fabricated in a nested 
(recurslive calling) structure. 


(cf Don't those algorithms, which are the Souls of the routines, Seem to have been 
born to be in a nested structure? 


1.23 Avoiding Runtime Error in Case of DefticienVNonadmissible Input Argu- 
ments 
(a) Consider the MAILAB routine“rotation_r(x,M)”， which you made 
in Problem 1.1S(h). Does it work Somehow when the user gives & 
negative integer as the Second input argument M ? Inot, add a statement 
So that it performs the rotation left by 一 M Samples for M < 0，Ssay， 
Imaking 


rotate_r([12345],-2) = [34512] 


(b) Consider the routine “trpzds(f,aib;N) ”in Section 9.0，which com- 
putes the integral of function f over [a, b] by dividing the integration 
interval into N Sections and applying the trapezoidal rule. If the user 
tries to use it without the fourth input argument N，will it work? If not， 
make it work with N = 1000 by default even without the fourth input 
argument N. 
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function INTf = trpzds(f,ayb;N) 
s%integral of f(x) over [a,b] by trapezoidal rule With N segments 
if abs(b - al) < eps | N<= 0，INTf = 0; return; end 


h= (bb- a)/NiXx=a+[0:N]*hi 
fx = feval(f,Xx); %5Vvalues of f for all nodes 
INTf = hx*((fx(1)+ fx(N+ 1))/2 + Sum(fx(2:N))); #%Eq.(5.6.1) 





Parameter Passing through varargin 


Consider the integration Toutine “trpzds(f,ab,N) ”in Section 9$.0. Can 
you apply it to _ compnute the integral of a_ function with some parame- 
ter(S)，like the “Bessel_integrand(x,beta,k)” that you defined in Prob- 
lem 1.21? 斑 not modify it so that it works for a function with Some param- 
eter(S) (See Section 1.3.0) and save lt in the M-file named "trpzds_par.m . 
Then replace the "quad() statement in the program “nm1p21b” (introduced 
in P1.21) by an appropriate "trpzds_par() ”statement (with N 王 1000) and 
run the program,， What ls the discrepancy between the integration Tesults 
obtained by this routine and the recursive computation based on Problem 
1.21.4(b)? Is it comparable with that obtained with “quad() ?7 How do you 
compare the running time of this routine with that of “quad() 7? Why do 
you think it takes so much time to execute the “quad() ”routine? 


Adaptive Input Argument to Avoid Runtime Error in the Case of Different 
Input Arsguments 


Conslider the integration routine "trpzds(f,ajb;,N) ”in Section 9.0. Isome 
User tries to _ use this routine with the following statement，will 站 
WorKk? 


trpzds(f,[a b],N) OT trpzds(f,[a b]) 


H 开 not, modify it so that it works for such a usage (with a bound vector as 
the second Input argument) as well as for the standard usage and Save it in 
the M-file named 'trpzds_bnd.m. Then try it to fnd the intergal of e- 
for [0,100] by typing the following statements in the MAILAB command 
window. What did you get? 


>>ftn=inline(' exp(-t) ，tt'); 
>>trpzds_bnd(ftn,[0 100] ,1000) 
>>trpzds_bnd(ftn, [0 100]) 


CtFEI(Continuous-Time Fourier Transform) of an Arbitrary Signal 


Consider the following deftinitions of CtFT and ICtETUmnverse CUFET) [W-4]: 


X(w) 一 Fxz(D} 一 三 x(eodf: CUET (P1.20.1a) 


Oo 


x() = Fr-IX(o)} = 元 三 X(w)e7erdw: ICtFT (PP1.26.1b) 


(3) 


(b) 
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Similarly to the MAILAB routine“CctFT1(x,Dt,w)”computing the 
CtFT (P1.20.1a) of x(t) over [-Dt,Dt ] for w make a MATILAB rou- 
tine“ICtFT1(X,Bw,t)”computing the ICUFT (P1.26.1b) of X(w) over 
[-Bw，Bw] for t. You can choose whatever integral routine including 
trpzds_par()” (Problem 1.24) and “quad() ,considering the running 
tme. 


The following program “nm1p26” finds the CUFT of a rectangular pulse 
(with duration [一 1,1]) deftned by "rpDt() ”for w= [一 0r, 十 or] andthe 
ICtFT of a sinc Spectrum (with bandwidth 2r) defined by “sincBw()” 
for ! = [一 9, 十 9]. After having Saved the routines into Mr-files with the 
appropriate names，run the program to see the rectangular pulse，its 
CtFT spectrum，a Sinc Spectrum，and jlts ICtFET. I it doesen't work， 
modify/supplement the routines So that you can Terun it to see the 
Signals and their Spectra. 


function Xxw = CtFT1(x,Dt,w) 

X_ejkwt = inline([Xx '(t).*exp(-j*wxt) ]，t' ，wW' ); 
Xw = trpzds_par(x_ejkwt,-Dt,Dt,1000,w) ; 

%5Xw = quad(x_ejkwt,-Dt,Dt,[], 0,w); 


function xt = ICtFT1(X,Bwy,t) 


function x = FrDt(t) 


(-D/2 <= t&t<= D/2); 


function X = SincBw(w) 


2*pi/Bx*Ssinc(WwW/B) ; 


%nm1p26: CtFT and ICtFT 
Clear，CclLf 
global B D 
%CtFT of a Rectangular Pulse Function 
[-50:50]/10; %time Vector 
[-60:60]/10*pi;i %frequency Vector 
1; #%Duration of a rectangular pulse rD(t) 
for kK= 1:length(w)，Xw(k) = CtFT1('rDt',Dx*5,W(k)); end 
Subplot(221)，plot(t,rDt(t)) 
SubplLlot(222)，plot(w,abs(Xw) ) 
%ICtFT of a Sinc Spectrum 
B = 2*pij; #%sBandwidth of a Sinc Spectrum sncB(w) 
forn= 1:1length(t)，xt(n) = ICtFT1(' SincBw' ,Bx*5,t(n) ); 
end 
Subplot(223)，plot(t,real(xt)) 
Subplot(224)，plot(w,sincBw(w) ) 


SYSTEM OF LINEAR 
EQUATIONS 





In this chapter,， we deal with Several numerical Schemes for solving a System of 
equations 
QHIX1 十 0127X2 十 :… 十 QINXN 一 DI 
021X1 十 022X2 十 … 十 Q2NXN 一 D2 
(2.0.1a) 


QGMIXI 十 QM2X2 十 … :十 GMNXN 一 DM 


which can be written in a compact form by using a matrix-vector notation as 


这 到 元 交 主 宙 C.0.1b) 
where 
02 aqaIN 覆 
| 
QM2 ， ， QMN | 人 


We will deal with the three caseS: 


() The case where the number (AM1) of equations and the number (W) of 
Unknowns are equal (M = W) so that the _ coefficient matrix 4MxN is 
Square. 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang,Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 人 儿 Sons, Inc. 
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(i) The case where the number (MT) of equations is Smaller than the number 
(V) of unknowns (AM < N) so that we might have to fnd the minimum- 
norm solution among the numerous Solutions. 

(ii) The case where the number of equations is greater than the number of 
Unknowns (M > N) so that there might exlst no exact Solution and we 
must fnd a solution based on global error minimization，like the“LSE 
(Least-squares error) Solution.” 


2.1 SOLUTION FOR A SYSTEM OF LINEAR EQUATIONS 


2.1.1 The Nonsingular Case (V = NM) 


I the number (4) of equations and the number (NW) of unknowns are equal 
(C = Nj, then the _ coefficient matrix 4 is square So that the solution can be 
WrTitten as 

x 一 4-!b (2.1.1) 


So long as the matrix 4 is not Singular. There are MAILAB commands for 
this job. 


>>A= [1 2;3 4];b= [-1i-1]j 
>>X = A^-1x*b %5or，X = inv(A)*b 
Xx = 1.0000 
-1.0000 


What if 4 is square, but Singular? 


>>A= [12;24];b= [1]; 
>>X = A^-1*b 
Warning: Matrix is Singular to working precision. 
Xx = -Inf 
-Inf 


This ls the case where Some or all of the rows of the coefficient matrix 4 are 
dependent on other rows and so the rank of 4 is deficient，which implies that 
there are Some equations equivalent to or inconsistent with other equations. 正 
we Iemove the dependent rows until all the (remaining) rows are Independent of 
each other So that 4 has full rank (equal to M), it leads to the case of M < N， 
which will be dealt with in the next Section. 


2.1.2 The Underdetermined Case (V < NM: Minimum-Norm Solution 


I the number (AM) of equations ls less than the number (W) of unknowns, the 
Solution ls not unique，but numerous. Suppose the M rowgs of the coefficient 
matrix 4 are independent. Then, any NM-dimensional vector can be decomposed 
into two components 

X 一 X+ 十 X- (2.1.2) 
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whpere the one ls in the row Space 及 (4) of 4 that can be expressed as a linear 
combination of the M row vectors 


x+ 一 47w (2.1.3) 


and the other is in the null space W(4) orthogonal(perpendicular) to the row 
Spacel so that 
4x 一 0 (2.1.4) 


Substituting the arbitrary N-dimensional vector Tepresentation (2.1.2) into 
Eq. (2.0.1) yields 


- 2 447e =b (2.1.5) 


4G+ 二 X-) 一 447w 十 4x 
Since 447 is supposedly anonsingular M x M matrix resulting from multiplying 
an MXxANmatrxby an NxAMmatrix， wecan solve this equation for w to get 


o" = [44 中 -mb (2.1.0) 
Then, substituting Eq. (2.1.6) into Eq. (2.1.3) yields 
xo+ 2) 4T7ao “9 47[447]-Ib 从 二 放 


This satisfies Eq. (2.0.1) and thus quajlifies as its Solution. However, it ls far 
from being a unique solution because the addition of any vector X (in the 
null space) satisfying Eq. (2.1.4) to X2+ still satisfies Eq. (2.0.1) [as seen from 
Eq. (2.1.3)], yielding infinitely many Solutions. 

Based on the Principle that any one of the two perpendicular legs ls Shorter 
than the hypotenuse in aright-angled triangle, Eq. (2.1.7) is believed to represent 
the 72ipizpzz7z-7no7z Softion. Note that the matrix 47[447]-1 is called the right 
pseudo- (generalized) inverse of 4 (See item 2 in Remark 1.1). 

MAILAB has the pinv() command for obtaining the pseudo-inverse. We 
can use this command or the slash(/) operator to find the minimum-norm solu- 
tion (2.1.7) to the System of linear equations (2.0.1). 


>>A= [12];b= 3 
>>X = pinv(A)*b %5x = A'*(A*A' )^- 1*b or eye(Size(A,2))/Ax*b，equivalent1y 
Xx = 0.6000 
1.2000 


了 emark 2.1. Projection Operator and Minimum-Norm Solution 


1.， The solution (2.1.7) can be viewed as the projection of an arbitrary solution 
X” onto the row Space 及 (4) of the coefficient matrix 4 Spanned by the 


1 See the website @http:/www.psc.edu/~burkardypaperslinear_glossary.html 
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IOwW vectors. The remaining component of the Solution X? 


xo- 一 xxXo+ 一 x 一 4T[A47]-Ib =xo 一 4T7[447]-14xo 
一 [T 一 47[44-14]x2 


is in the null space W(C4), since it satisfies Eq. (2.1.4). Note that 
P=[IT 一 47[447]-14] 


is called the projection operator. 


2. The solution (2.1.7) can be obtained by applying the Lagrange multiplier 
method (Section 7.2.1) to the constrained optimization problem in which 
we must find a vector x minimizing the (squared) norm ||xl|> subject to the 
edquality constraint 4X 一 b. 


Eq.(7.2.2) 1 


Min 1(Cx, 入 ) 3x 上 一 和 (4x 一 b) = 了 zx 一 和 (4x 一 b) 


By using Eq. (7.2.3)，we get 


8 
二 一 X 一 4 人 = 0; x=47 人 = 47[447]-b 
玉 


8 
= 一 4x--b=0; 447 = b; 入 = [44 站 -mb 


了 kxample 2.1. Minimum-Norm Solution. Consider the problem of solving the 
equation 


[1 | | 二 4x=b，where4=[1 2]，b=3 (2.1.0 
之 


This has infinitely many solutions and any X 一 [xl xz2]7 satisfying this 
equation，or, equivalently， 


1 3 
X1 十 2x2 一 3; X2 一 一 52 十 (E2.1.2) 


is a qualified solution. Equation (E2.1.2) describes the solution Space as depicted 
in Fig. 2.1. 

On the other hand, any vector in the row Space of the coefficient matrix A 
can be expressed by Eq. (2.1.3) as 


x+ 一 47w= 目 Q (a is a scalar, since M = 1) (上 2.1.3) 
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3/5, 6/5) = xo+ 
(1, 1) =X9 


Solution 








Figure 2.1 Aminimum-norm solution， 


and any vector in the null Space of 4 can be expressed by Eq. (2.1.4) as 
于 2 上 去 
4X 一 [1 | | 沽 和 (E2.1.4) 
XD 分 


We use Eq. (2.1.7) to obtain the minimum-norm solution 


-1 
xo+ 47[4A47]-b = | (0 2] 本 3 二 3 四 四 人 3 (E2.1.5) 


Note from Fig. 2.1 that the minimum-norm solution x2+ is the intersection of 
the solution Space and the row Space and ls the closest to the origin among the 
vectors in the solution Space. 


2.1.3 The Overdetermined Case (W > NM:LSE Solution 


HI the number (M) of (independent) equations is greater than the number (W) 
of unknowns, there exists no Solution Satisfying all the equations stricty. Thus 
we try to find the LSE (least-squares error) Solution minimizing the norm of the 
(inevitable) error vector 

e 一 4x 一 b (2.1.8) 


Then, our problem is to minimize the objective function 


J 一 划 el= 吉 hx 一 bl2=J4x 一 bl7[4x 一 b] (2.1.9) 
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whose solution can be obtained by setting the derivative of this function (2.1.9) 
With respect to X to Zero. 


二 7 = 47[4x-b]= 0; x2 二 [474]-147Pb (2.1.10) 
尺 


Note that the matrix 4 having the number of rowsgs greater than the number of 
columns (AM > NW) does not have its Inverse, but has lts left pseudo (generallized) 
inverse [474]-147 as long as 4 is not rank-deficient 一 that is, all of its columns 
are independent of each other (See item 2 in Remark 1.1). The left pseudo-inverse 
matrix can be computed by using the MAILAB command pinv(). 

The LSE solution (2.1.10) can be obtained by using the pinv() command or 
the backslash 人) operator. 


>>A= [1; 2]jb= [2.1; 3.9]; 
>>X = pinv(A)*b %AN\b or X = (A'*A)^-1*A'*b，equivalent1y 
x = 1.9800 


function xX= Lin eq(A,B) 
s%This function finds the Solution to Ax 
[M,N] = Size(A) ; 
if Size(B,1) ~= M 
error(' Incompatible dimension of AandBain lin edq()! ) 
end 


ifM==NXxX=A-1*B; %x = inv(A)*B or gaussj(A,B); %5Eq.(2.1.1) 
elseif M< Ns%Minimum-norm Solution (2.1.7) 
X = pinv(A)*B; %5A'*(A*A' )^-1*B; or eye(Size(A,2) )/A*B 
elSse %LSE Solution (2.1.10) for M > N 
X = pinv(A)*B; 5%5(A'*A)^-1*xA'*B or x = ANB 
end 





The above MAILAB routine lin _ eq() is designed to solve a given Set of 
equations，covering all of the three cases in Sections 2.1.1, 2.1.2, and 2.1.3. 


(cf) The power of the pinv() command is beyond our imagination as you might have 
felt in Problem 1.14. Even in the case of M < N, it fnds us a LS Solution it the 
equations are inconsistent. Even in the case of M > N, it fnds us a minimum-norm 
solution if the equations are redundant. Actually, the three cases can be dealt with 
by a single pinv() command in the above routine. 


2.1.4  RLSE (Recursive Least-Squares Estimationm) 


In this section we will see the So-called RLSE (Recursive Least-Squares Esti- 
mation) algorithm，which is a recursive method to compnute the LSE solution. 
Suppose we know the theoretical relationship between the temperature 态 ] and 
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the reslstance R[S] of a resistor as 
clt 十 cz 一 玉 


and we have lots of experimental data {(0, RD (PR2) (大 , RE collected 
up to timeK. Since the above equation cannot be Satisfied for all the data with any 
value of the parameters cl and cz?，we Should try to get the parameter estimates 
that are optimal in Some Sense. This corresponds to the overdetermined case dealt 
with in the previous Section and can be formulated as an LSE problem that we 
must Solve a set of linear equations 


力 1 Ri 
4kXK 久 bp， where 4/ 一 灰 村 JR 9 ，and by 一 

2 . 

丸 1 民 5 


for which we can apply Eq. (2.1.10) to get the solution as 
太一 [47407-147bx (2.1.11) 


Now, we are given a new experimental data (大 +1，Rk+l) and must fnd the 
new parameter estimate 





xeH = [4 和 4 Abei (2.1.12) 
with 
四 1 及 1 
4t+1l 三 ， Xk+1 一 区 呈 ， and btk+l 三 R， 
加 1 1 RE+1 


How do we compnute this? I we discard the previous estimate Xk and make direct 
use of Eq. (2.1.12) to compute the next estimate Xk+l every time a new data pair 
is avallable, the Size of matrix 4 will get bigger and bigger as the data pile up， 
eventually defying any powerful compnuter in this world. 

How about updating the previous estimate by just adding the correction term 
based on the new data to get the new estimate? This ls the basic idea of the 
RLSE algorithm，which we are going to trace and try to understand. In order to 
do So, let us define the notations 


人 4 灰 +1 bx 了 一 1 
人 王 让 汪 二 ， hb 三 ，and 己 =[4, 4 
人 H1 | 人 | 人 1 | 1 | 人 1 有 一 [4 4 
(2.1.13) 
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and see how the inverse matrix Pi is to be updated on arrival of the new data 
(+1， Re 


于 
芭 人 
Pa = [404e = | ak+1] 罗 | 
H1 





= [41( 4 十 arHat = [PITarat .114) 
(Matrix Inversion Lemma in Appendix B) 


PH = 大 一 PakrHi[at Peak 十 1 at 庆 (2.1.19) 


It is interesting that [az Peak+i 十 ]] is nothing but a scalar and So we do 
not need to compnute the matrix inverse thanks to the Matrix Inversion Lemma 
(Appendix B). It is much better in the computational aspect to use the recursive 
formula (2.1.15) than to compute [41 AH directly. We can also write Eq. 
(2.1.12) in a recursive form as 


C.112,14) 二 C.1.13) 记 bx 
Xk+H1 三 PHI4DIibrl 三” Pil4 akrl] 攻 
十 


]) 


C.1. 
= Pi[41 pb 二 aiRe “三 ”PiA4T4EXe 二 aiRei] 





三 ”Pi [(444Hl 一 atLla7 Xe 十 akHiRei] 








三 ”Pi [Paxx 二 akH1aA HT 十 acH1IARk+HI 
XEH 一 kt 十 PHIakHICOREH 一 af xb) (2.1.10) 


We can use Eq. (2.1.15) to Tewrite the gain matrix 户 +1ak+l premultiplied by 
the “error to make the correction term on the right-hand side of Eq. (2.1.16) as 


C.1L1 


5) 
天 1 三 Pak+Hl [ 户 一 Pakti[a7 Peakri 二 ]H] Ia Pi]ak+i 








二 -akHIL 一 [at Pearl 十 ]]-1a7 Pak+l] 


一 PakHi[at Peaktl 十 1 {[at Peak 十 ]] 一 at PearHl} 











KLl = akti[ati Pearl 十 1] 一 C.1.17) 

and substitute this back into Eq. (2.1.13) to write it as 
Pa = 挛 一 Rati (2.1.18) 
The following MAILAB routine“rlse_online()”implements this RLSE 


(Recursive Least-9quares Estimation) algorithm that updates the parameter 
estimates by using Eqs. (2.1.17), (2.1.16), and (2.1.18). The MAILAB program 
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“do_rlse.m” updates the parameter estimates every time new data arrive and 
compares the results of the on-line processing with those obtained by the off-line 
(batch job) processing 一 that is, by using Eq.(2.1.12) directly. Noting that 


。the matrix [47 44] as well as bx consists of information and is a kind of 
Squared matrix that ls nonnegative, and 

。[47 46] wil get larger, or equivalently, 记 = [47 4- will get smaller and， 
consequently, the gain matrix KK will get Smaller as valuable information 
data accumulate， 


one could understand that 户 is initiallized to a very large identity matrix，Since 
no information ls available in the beginning. Since a large/small 已 makes the 
Correction term on the right-hand slide of Eq. (2.1.16) large/small， the RLSE 
algorithm becomes more conservative and reluctant to learn from the new data 
as the data pile up, while it is willing to make use of the new data for updating 
the estimates when it is hungry for information in the beginning. 


function [x,K,P] = rlse_ online(aT_kl,b_k1,x,P) 
K = PxaT_k1'/(aT_k1x*PxaT_k1'+1); %Eq.(2.1.17) 
x = Xx +Kx(b _k1-aT_k1x*x); %Eq.(2.1.16) 

P = P-Kx*aT_klx*P;  %Eq.(2.1.18) 


%5do_Fl1Sse 
Clear 
XO [2 1] ' ; #%The true value of unknown coefficient vector 
NA Jength(xo ) ; 
ZeroS(NA,1); P = 100*eye(NA;NA) 
for k = 1:100 
A(k,:) = [kx*0.01 1]; 
b(k,:) = A(k,:)*xo +0.2x*rand ; 
[x,K,P] = Prlse online(A(k,:)，,b(k,:),，x,P); 
end 
X 
A\b 


he final parameter estimate 
or comparison with the off-lLine processing (batch job ) 





% 攻 
% 下 


2.2 SOLVING A SYSTEM OF LINEAR EQUATIONS 


2.2.16 ”Gauss Elimination 

For Simplicity，we assume that the coeffticient matrix 4 in Eq. (2.0.1) 1 a non- 

Singular 3 x 3 matrix with M = N = 3. Then we can write the equation as 
QIX1 十 Ql2X2 十 Q13X3 一 D1 (2.2.0a) 
C21X1 十 022X2 十 023X3 一 D2 (2.2.0b) 


0Q31X1 十 032X2 十 033X3 一 D3 (2.2.0c) 
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First, to remove the xl terms from equations (2.2.0.17) other than (2.2.0.a), we 
Subtract (2.2.0a)xanmnl/ail from each of them to get 


am + 鸣 吉 += 罗 全 
022 z2 十 03 妇 一 及 (2.2.1b) 
0 和 2 十 03 3 一 扩 0 
with 
a0 一 am pb = 名， formn=l12.3 (2.2.2a) 
0 网 二 0 一 (anijalil)a， 二 轴 一 (ni/ai)b form 一 2.3 


(2.2.2b) 
We call this work "pivoting at alil” and call the center element al11 a “plivot . 
Next to remove the x2 term from Eq. (2.2.1c) other than (2.2.1ab)，we Sub- 
tract (2.2. lb)xao/a 刀 (on 一 一 3) from it to get 


ax 击 al2xo 才 al3x3 一 一 人 ) (2.2.3a) 
4 和 十 史记 = 区) (2.2.3b) 
ax23 一 09 (2.2.3c) 


with 


人 = 人 -Ca 和 = 史 -Ca 和 各 mn 
(2.2.4) 
We call this procedure “Gauss forward elimination ”and can generalize the updat- 
ing formula (2.2.2)/(2.2.4) as 


0 一 ak 一 (GD/a0 Da form 一 大 十 1 大 十 2,...，M (02.2.5a) 


11 用 Ci 


p =DUD 一 (Da D)pe for 六 一 大 十 1 十 2,...,M (0.2.5b) 


After having the triangular matrix-vector equation as Eq. (2.2.3), we can Solve 
Eq. (2.2.3c) first to get 
一 0/a 归 (2.2.6a) 


and then Substitute this result into Eq. (2.2.3b) to get 
总 一 (0 一 az)/a 纪 (2.2.6b) 


Successively，we Substitute Eqs. (2.2.0a,b) into Eq.(2.2.3a) to get 


总 
xl 一 攻 芭 | /af9) (2.2.6c) 
站 
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We call this procedure “backward Substitution”and can generallize the solution 
formula (2.2.6) as 


M 
一 (or 一 》， 人 /ae ) for1 一 MAM 一 1... ,1 
7 一 1 十 1 
(2.2.7) 
In this way, the Gauss elimination procedure consists of two StepSs, namely， 
forward elimination and backward substitution. Noting that 


。this procedure has nothing to do with the specific values of the unknown 
variapble xmw s and involves only the coefficients, and 


。the formulas (2.2.9a) on the coefficient matrix 4 and (2.2.Sb) on the RHS 
(right-hand side) vector b conform with each other， 


we will augment 4 with b and put the formulas (2.2.3a,b) together into one 
framework when programming the Gauss forward elimination procedure. 


2.2.2 ”Partial Pivoting 


The core formula (2.2.5) used for Gauss elimination requires division by aWD) 


at the Kth stage，where is the diagonal element in the kth row. What 让 


QI D) 一 0 In such a case, it is customary to Switch the kth row and another row 
below it having the element of the largest absolute value in the Kth column. This 
procedure, called “partial pivoting ,is recommended for reducing the round-off 
efror even in the case Where the Kth pivot 人 is not zero. 


Let us consider the following example: 


0 1 1 X1 D1 王 没 
2 1 兴 二 5 三 但 (2.2.8) 
1 1 一 1 3 D3 一 1 


We construct the augmented matrix by combining the coeffticient matrix and the 
RHS vector to write 


0 
QG21 0G227 023 D2 一 |2 -1 -1 0|: 72 (2.2.9) 
031 032 033 2D3 1 


and apply the Gauss elimnination procedure. 

In the stage of forward elimination，we want to do pivoting at al1，bnut all 
cannot be used as the pivoting element because it ls zero. So we Switch the first 
Iow and the second row having the element of the largest absolute value in the 
first column. 
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(D) () () (D) 

11 42 43 帮 

人 
1 1 1 1 1 

客人 1 1 -1 二 守 


Then we do plivoting at da 1) by applying Eq. (2.2.2) to get 


-[ 鸣 喝 别 吕 

癌 - 史 加 xz 和 | 哆 哆 哆 兄 

更- 喝 鸡 >x” 党 喝 虽 你 
2 -1 -1 01: ro 
=|0 1 1 2|: 7 
0 3/2 -1U2 1 :， 72 


(2.2.10a) 


(2.2.10b) 


人 2，wWe Switch the second row and the third row 


Here, instead of pivoting at a27 ， 


having the element of the largest absolute value among the elements not above 


2) in the second column. 


522 
(3) (3) (G) G) 二 AH) 
da ap a3 全 几 1 1 0 | : 人 
0 01]=|0 32 -12 1 :各 
3 3 3 3 3 
a9 auag aa 29) 0 1 1 9 


(2.2.10c) 


And we do pivoting at a2) by applying Eq. (2.2.4) 一 more generally，Eq. 


(2.2.9) 一 to get the upper-triangularized form: 


一 [ 史 唤 唤 蝇 

人 ”| 唱 哆 喝 邓 

虽 - 喝 唱 x 昌 [党 鸣 唤 加 
2 -1 -l1 0 了 

=|0 3/2 -L2 1 |: 7 


0 0 43 43] :7 各 


(2.2.10d) 


Now, in the stage of backward Substitution，we apply Eq. (2.2.6), more gen- 


erally，Eq. (2.2.7) to get the final solution as 
盖 久 /aa = (4/3) 人 3) = 1 


4 不 4 
ji = (人 一 ama)/a2=( 


一 (CIL 力 xD/AG3/ 轨 =1 


(2.2.11) 


3 
X1 一 必 宕 2 /al =(0-(-Dxl-(-DxlD/2=1 
7 一 2 


pi xx]=[L 1 


(2.2.12) 
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Let us consider another System of equations. 


] 0 1 愉 1 DPI 三 六 
1 1711=| 5=3 (2.2.13) 
1] 一 1 1 人 | D3 三 沁 


We construct the augmented matrix by combining the coeffticient matrix and the 
RHS vector to write 


da 0 43 1 012|: 方 
CC21 0G22 023 D2 一 1 1 3 72 (2.2.14) 
031 032 033 2D3 1 一 1 1 7 


and apply the Gauss elimination procedure. 
EFirst, noting that all the elements in the first column have the same absolute 
value and So we don't need to Switch the rowS,， we do pivoting at aq11. 


(D) (D) (D) (D) (D) 
a1 aq7 aa 全 ] 0 1 2|: 六 
ao |=|o 10 13 7 (2.2.15a) 
1 1 1 1 1 
呈 0 2 中 : 0 -1 0 -1 |: 科 ) 


Second, without having to Switch the rowS，we perform pivoting at 0 包 . 


AN 

各 | 人 有 有 史 |=|oiol: 各 
| 
(2.2.15b) 


Now, we are at the Stage of backward Substitution, but da0， which is Supposed 
to be the denominator in Eq. (2.2.7), ls zero. We may face Such a weird Situation 
of zero division even during the forward elimination process where the pivot is 
Zero; besides, we cannot fnd any (nonzero) element below it in the same column 
and on its right in the Same IOw except the RHS element. In this case，we 
cannot go further. This implies that Some or all rows of coefficient matrix 4 are 
dependent on others，corresponding to the case of redundancy (infinitely many 
Solutions) or inconsistency (no exact Solution). Noting that the RHS element 
of the zero row in Eq. (2.2.13.2) ls also zero，we Should declare the case of 
redundancy and may have to be Satisfied with one of the infinitely many solutions 
being the RHS vector as 


px 2]=[ 02 20]=[D2 10] (2.2.16) 


Furthermore, f we remove the all-zero Tow(S), the problem can be treated as an 
underdetermined case handled in Section 2.1.2. Note that, 夺 the RHS _ element 
Were not zero，we would have to declare the case of inconsistency,，as will be 
jllustrated. 

Suppose that p1 = 1 in Eq. (2.2.14). Then, the Gauss elimination would have 
proceeded as follows: 
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1 011 1 011 1011 
1 113|>|lo0 102|>|010 :> (2.2.17) 
1 -1 11 0 -1 0 0 000 2 


This ended up with an all-zero TOwW except the nonzero RHS element，corre- 
Sponding to the case of inconsistency. So we Imust declare the case of mo exact 
solution ”for this problem. 

The following MAILAB routine “gauss()”implements the Gauss elimination 
algorithm, and the program “do_gauss” is designed to solve Eq. (2.2.8) by using 
“gauss()”. Note that at every pivoting operation in the routine“gauss()”, the 
pivot row is divided by the pivot element so that every diagonal element becomes 
one and that we don't need to perform any computation for the kth column at 


the kth stage, Since the column is Supposed to be all zeros but the kth element 
(O 
1. 


function X = gauss(A,B) 
%The Sizes of matrices A,B are Supposed to be NA Xx NA and NA x NB. 
%This function SoLVeSs Ax B by Gauss elimination algorithm . 
NA = Size(A,2); [NB1,NB] Size(B) ; 
If NB1 ~= NA，error('Aand B must have compatib1le dimensions ' ); end 
N=NA+ANB; AB = [AL1:NA,1:NA) B(1:NA,1:NB)]; % Augmented matrix 
epss = epSsx*ones(NA,1) ; 
for k = 1:NA 
%Scaled Partial Pivoting at AB(k,k) by Edq.(2.2.20) 
[akx,kx]l = max(abs(AB(k:NA,k))./ ... 
max(abs([AB(kK:NA,k + 1:NA) epss(1:NA -k+ 1)] )) ); 
if akx < eps，error(' Singular matrix and No unique Solution' ); end 
mx=kr+ kx - 1 
if kx>155 Row change If necessary 
tmp_row = AB(k,k:N); 
AB(k,k:N) = AB(mx,k:N); 
AB(mx,k:N) = tmp_row; 
end 
% Gauss forward elimination 
AB(k,k+ 1:N) = AB(k,k+1:N)/AB(k,k); 
AB(k,k) = 1; s%make each diagonal element one 
form=k+1: NA 
AB(m,k+1:N) = AB(m,k+1:N) - AB(m,Kk)*AB(k,k+1:N); %5Eq.(2.2.5) 
AB(m,k) = 0; 
end 
end 
s%backward Substitution for a upper-triangular matrix eqation 
s#% having all the diagonal elements equal to one 
X(NA,:) = AB(NA,NA+1:N) 
for m= NA-1: -1:1 
x(m,:) = AB(m'NA + 1:N)-AB(m'm+ 1:NA)*x(m + 1:NA,:); #%Eq.(2.2.7) 


x = gauss(A,b) 
xl1 = AN\b %for comparison With the result of backslash operation 
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(cf) The number of foating-point multiplications required in this routine “gauss() ”is 


N4 AN4 一 1 
>》 (NM 一 K+DWMAT+TNB 一 及 十 NA 一 大 十 二 十 》 (NA 一品 NB 
大 =1 K=1 











NA4 N4 NA4 
= 》 KK+TNB -1D)-NB》 +》 NA NB 
天 三 | 天 三 让 
1 1 
一 EC 十 DN4(CNM4 十 1 一 NA(VA4 十 ]) 十 N42NB 


1 
一 3 了 MACVA 十 DOMA 一 1 +NA2NB 


1 
包 了 NM4- for MA > NB (2.2.18) 


where NM4 jls the Size of the matrix 4, and MB is the column dimension of the RHS 
matrix 妃 . 


Here are Several things to note. 


了 R 了 emark 2.2. Partial Pivoting and Undetermined/Inconsistent Case 


1. In Gauss or Gauss-Jordan elimination，Some row Switching is performed 
to avoid the zero division. Even without that purpose, it may be helpful 
for reducing the round-off error to fx 


Max{lamkl, KE 三 1 三 M} (2.2.19) 


as the pivot element in the kth iteration through Some row Switching, which 
is called "partial pivoting.”Actually, it might be better off to fx 


| (2.2.20) 
Max{lam|l ,上 三 站 三 MI) 


as the pivot element in the Kth iteration，which is called “scaled partial 
pivoting” or to do column Switching as well as Tow Switching for choosing 
the best (largest) pivot element，which is called “full pivoting.”Note that 
if the columns are Switched, the order of the unknown variables should be 
interchanged accordingly. 

2. What if some diagonal element akk and all the elements below it in the 
Same column are Zero and，besides, all the elements in the row including 
ak are alSo Zero except the RHS element? It implies that Some or all 
IOowSs of the coefficient matrix 4 are dependent on others，corresponding 
to the case of redundancy (infinltely many solutions) or inconsistency (no 
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exact Solution). If even the RHS element is zero, it should be declared 
to be the case of redundancy. In this case，we can get rid of the all-zero 
Iow(S) and then treat the problem as the underdetermined case handled in 
Section 2.1.2. Hthe RHS element is only one nonzero in the row, it Should 
be declared to be the case of inconsistency、. 


卫 xXample 2.2. Delicacy of Partial Pivoting. To get an actual feeling about the 
delicacy of partial pivoting，consider the following systems of linear equations， 
which apparently have x2 = [1 1]7 as their solutions. 


105 1 1 十 10-5 
(a) Aix= bl with Al 中 1 吕 ， bi = | 10u+1 (上 2.2.1) 


Without any row Switching, the Gauss elimination procedure will fnd us 
the true solution only 这 there ls no quantization error. 


10-5 1 1+10-5 
[4 bi] | 1 1011 1011 十 1 


forward backward 
elimination 1 1015 105 十 1 substitution 1 
0 104 一 105 101 -105 三 | 休 


But, because of the round-off error, it will deviate from the true Solution. 


forward 1 1015 = 9.999999999999999e+014 105 二 1 工 = 1.000000000000001e+015 





elimination 
0 101 一 105 101 二 1-(105 一 JTJ) 
= 一 9.998999999999999e+014 = 一 9.999000000000000e+014 
backward 
substitution ，_ | 8.750000000000000e-001 
> X 二 | 1.000000000000000e+000 


HI we enforce the Strategy of partial pivoting or Scaled partial pivoting, the 
Gauss elimination procedure will give us much better Tesult as follows: 


IOW SWap 1 1011 1011 于 1 
[pb 一 攻 > 1 1+10-5 


forward 
elimination 1 1011 = 1.000e+011 1011 十 1 = 1.000000000010000e+011 
0 1--10-4= 9.999e-001 9.999000000000001le-001 
backward 
Substitution 9.99984741210937Se-001 
”| 1.000000000000000e+000 


_14.6 一 14.6 
人 和 坟 三 机 二 =| 2 | 交 | | (E2.2.2) 
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Without partial plivoting, the Gauss elimination procedure will give us a 
quite good result. 


[4 Di]= 1 10!6 = 3.981071705534969e+014 10146 十 1 = 3.981071705534979e+014 
| 人 0.018928294465030e+014 0.018928294465030e+014 


吧 长 3.981071705534969e+014 | 
0 1 1 


backward 
Substitution | 1 | 
一 一 一 一 一 X 一 1 
But, 寺 we exchange the first row with the Second row having the larger 
element in the first column according to the strategy of partial plivoting, the 


Gauss elimination procedure will give us a rather Surprisingly bad result 


as follows: 
本 1 105 = 1.000000000000000e+015 105 二 1 = 1.000000000000001e+015 
S pplng 
0 1 一 1015 10-14.6 1 十 10-14.6 汪 (1 阅 100) 10-14.6 
妇 d 三 
于 = 一 1.5118864315095819 = -1.5118864315095821 
backward 
Substitution 二 0.7500000000000000 
”| 1.0000000000000002 


One might be happy to have the Scaled partial pivoting Scheme 
[Eq. (2.2.20)]，which does not Switch the rows in this case，Ssince the 
relative magnitude (dominancy) of all in the first row is greater than that 
of azl in the second row, that is,，10-146/1 > 17105. 


105 1 105+1 
(c) Asx= by with A3 = | oz] ，bs = |， | (E2.2.3) 


With any plivoting Scheme，we don't need to Switch the rowS，since the 
relative magnitude as well as the absolute magnitude of all in the first row 
is greater than those of al in the second row. Thus, the Gauss elimination 
procedure will go as follows: 


forward 


elimination 1 1.000000000000000e-015 1.000000000000001le+000 
0 1.311886431509582e-015 ”1.33220760295350188e-015 
backward 
Substitution 1.000000000000000 
”| 0.811955724875121 


(cf) Note that the coefticient matrix，43 is the Same as would be obtained by applying 
the full pivoting Scheme for 4， to have the largest pivot element. This example 
implies that the Gauss elimination with full pivoting Scheme may produce a worse 
result than would be obtained with scaled partial pivoting Scheme. As a matter of 
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factor，we cannot Say that Some pivoting Scheme always yields better solution than 
other pivoting Schemes, because the result depends on the random round-off error as 
well as the pivoting scheme (See Problem 2.2). But in most cases, the Scaled partial 
pivoting Shows a ITeasonably good performance and that is why we adopt it in our 
Ioutine“gauss ()”. 


了 emark 2.3. Computing Error, Singularity, and HI-Condition 


1. As the Size of the matrix grows, the round-off errors are apt to accumu- 
late and propagated in matrix operations to Such a degree that zero may 
appear to be an absolutely small number,， or a nonzero number very close 
to zero may appear to be zero. Therefore, it ls not So Simple a task to 
determine whether a zero or a number very close to zero ls a real zero Or 
Dot， 

2. It ls desirable，but not So easy, for us to discern the case of Singularity 
from the case of ill-condition and to distinguish the case of redundancy 
from the case of inconsistency. In order to be able to give such a qual- 
itative judgment in the right way based on Some quantitative analysis， 
we Should be equipped with theoretical knowledge as well as Practical 
experience. 

3. There are several criteria by which we judge the degree of il-condition， 
such as how discrepant 44-1 is with the identity matrix，how far 
det{(4}det4-1 stays away from one(1), and so on: 


世人 一 11-1 了 -1、 了 
44 ”= 1/， [4 ] = 人 4， det(4)det(4 ) = 三 1 (2.2.21) 


The MAILAB command cond() tells us the degree of ill-condition for a 
given matrix by the Size of the condition number，which is deftined as 


cond(4) = 14 川 4- 虽 with 1 4 = largest eigenvalue of 47 4， 
ij.e., largest Singular value of 4 


卫 xample 2.3. The Hilbert matrix defined by 


4 = [am] = 三 | (E2.3) 


1 
1 十 1 一 1 


ls notorious for its -condition. 

We increase the dimension of the Hilbert matrix from N = 王 7to 12 and make 
use of the MAILAB commands cond() and det() to compnute the condition 
number and det(4)det(4-D in the MATLAB program“do_condition”. Espe- 
cially for N = 10, we will see the degree of discrepancy between 44-! and 
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the identity matrix. Note that the number RCOND following the warning message 
about near-Singularity or ill-condition given by MAILAB is a reciprocal condi- 
tion number,， which can be computed by the rcond() command and is Supposed 
to get close to L/0 for a well-/badly conditioned matrix. 


%do_condition.m 


hilb(6) ，Eq.(E2.3) 


Alm'N) = 1/(m+N- 1); end 
A(N,n) = 1/(N+n- 1); end 
= det(A)*det(A^- 1) 
fprintf( : cond(A) = %e， det(A)det(A^ - 1) = %58.6f\n' ，N，c，d) 
if N == = AxA^ - 1，end 
end 





>>do_condition 


N = 7: cond(A) = 4.753674e+008， det(A)det(A`-1) = 1.000000 

N= 8: cond(A) = 1.525758e+010， det(A)det(A`-1) = 1.000000 

N = 9: cond(A) = 4.931532e+011， det(A)det(A`-1) = 1.000001 

N = 10: cond(A) = 1.602534e+013， det(A)det(A`-1) = 0.999981 

AAI = 

1.0000 0.0000 -0.0001 -0.0000 0.0002 -0.0005 0.0010 -0.0010 0.0004 -0.0001 
0.0000 1.0000 -0.0001 -0.0000 0.0002 -0.0004 0.0007 -0.0007 0.0003 -0.0001 
0.0000 0.0000 1.0000 -0.0000 0.0002 -0.0004 0.0006 -0.0006 0.0003 -0.0000 
0.0000 0.0000 -0.0000 1.0000 0.0001 -0.0003 0.0005 -0.0006 0.0003 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 1.0001 -0.0003 0.0005 -0.0005 0.0002 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 0.0001 0.9998 0.0004 -0.0004 0.0002 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 0.0001 -0.0002 1.0003 -0.0004 0.0002 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 0.0001 -0.0002 0.0003 0.9997 0.0002 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 0.0001 -0.0001 0.0003 -0.0003 1.0001 -0.0000 
0.0000 0.0000 -0.0000 -0.0000 0.0001 -0.0002 0.0003 -0.0003 0.0001 1.0000 


之 
外 
一 
SR 


: cond(A) =5.218389e+014， det(A)det(A`-1) = 1.000119 
Warning: Matrix is close to Singular or badly scaled . 
Results may be inaccurate. RCOND = 3.659249e-017. 
> In C:NMATLAB\nma\do_condition.m at line 12 
N = 12: cond(A) =1.768065e+016， det(A)det(A`-1) = 1.015201 


2.2.3 Gauss-Jordan Elimination 


While Gauss elimination consists of forward elimination and backward sub- 
stitution as explained in Section 2.2.1，Gauss-Jordan elimination consists of 
forward/backward elimination，which makes the coefficient matrix 4 an identity 
matrix So that the resulting RHS vector will appear as the solution. 
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For Simplicity，we Start from the triangular matrix-vector equation (2.2.3) 
obtained by applying the forward ellimination: 


0 OO 0 70) 
aa1 42 43 DO 
0 oo 鸣 ) (人 .2.22) 


0 0 虽 如 


First we divide the last row by a 委 


(0) (0) (0) (0) 
ZI11 012 413 1 
0 do q 钨 2 (2.2.23) 


2) (2 
0 0 呈 =1 和 =521a9 
and Subtract (the third row xdaO D) (mo 一 一 12)) from the apove two rows to get 


(0) (0) [ (0) Za [ 
GT al 0 一 0 2 一 D 013 D5 


0 ao 0 =0 功 = 一 0 和 0 (2.2.24) 
0 0 潭 1 p 


Now, we divide the second row by da : 


2 (0) 


D 
411 212 0 人 
0 ad 节 一 1 0 2 < 有 1 /aa 局 (2.2.25) 
0 0 4 局 =1 2 


and Subtract (the second row xd 人 D Un 一 一 J)) from the above first row to get 


二 
0 10 D (2.2.26) 
0 0 1 DO 


Lastly，we divide the first row by aa) to get 
bD = .51an 


0 0 
1 0 D0 (2.2.27) 
0 1 


ce 
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which denotes a System of linear equations having an ldentity matrix as the 
coefficient matrix 
一 hu 一 TpG] [2] [ 17 
7X=b =[O 2D 0 ] 


and, consequently, take the RHS vector bl as the final solution. 

Note that we don't have to distinguish the two steps, the forward/backward 
elimination. In other words,， during the forward elimination，we do the pivot- 
ing operations in Such a way that the pivot becomes one and other elements 
above/below the pivot in the Same column become zeros. 

Consider the following System of linear equations: 


-1 -2 21]To | 
1 1 -zl=l 1 (2.2.28) 
1 2 -1|| az 2 


We construct the augmented matrix by combining the coefficient matrix and the 
RHS vector to write 


5 Di 一 1] 一 2 2， 二 1 2 
C21 0G22 023 D2 一 工 1 一 1 1 9 (2.2.29) 
CQ31 032 033 D3 1 2 一 | 2 4 


and apply the Gauss-Jordan elimnination procedure. 
First， we divide the first row Fi by all 三 一 L to make the new first row 7 
have the pivot af 一 1 and Subtract Cnml X r07(Om 一 2,3) from the second and 


third row r2 and 73 to get 


(D) 
1 


2 1 1 1 1 
六 (CD 一 [am ao ai 2 1 2 -2 1|: 
1 1 1 1 1 1 
记过 入 天 全 | 训 才 前 一 | 二 下 交合 三 王 和 庆 志 ， 
1 1 
六 -1xr” [中 克昌 | Lo oo1 吉 :次 
(2.2.30a) 
Then，we divide the Second row 7 by 0 包 一 一 ] to make the new Second TowW 
六 have the pivot a2) 一 1 and subtract da X 六 多 (1 一 1,3) from the first and 
third row r 人 ) and 7 to get 
1 和 
各 -2x 虽 >[ 哆 克 克昌 | [ooil: 兄 
1 2 2 2 ) 
PCD 一 | 加-10| 5 交 
2 2 2 2 2 
0 
(2.2.30b) 


Lastly，we divide the third row 唱和 by a5 一 1 to make the new third row 


7 have the pivot 0 电 一 1 and subtract aa 久 天 (110 一 1,2) from the first and 
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《 


Second TOW 7 多 and 7 人 to get 


2 3 

虽 -ox 昌 一 [ 哩 唱 唱 虽 | [eta]: 

/六 一 (CDxr 站 一 | 史上 |=|0101=z 和 
3 字 3 3 3 

史 了 


(2.2.30c) 
After having the identity matrix-vector form like this，we take the RHS vector 
as the Solution. 

The general formula applicable for Gauss-Jordan elimination ls the Same as 
Eq. (2.2.3)，except that the index Set ls 1 天 大 一 that 1$，all the numbers from 
1 一 ]to 7 一 AM except 7 一 K. Interested readers are recommended to make 
their own routines to Implement this algorithm (See Problem 2.3). 


2.3 INVERSE MATRIX 


In the previous section，we looked over some algorithms to solve a System of 
linear equations. We can use Such algorithms to Solve Several Systems of linear 
equations having the Same coefficient matrix 


A4XI1 一 b ， 4X> 一 b，， 人 4XNB 一 bx 
by putting different RHS vectors into one RHS matrix as 


4[xl X2…'XN] 一 [bl bz.…bwves]， 4X = 了 
C.3.0 
和 一 4-1B 


H 开 we Substitute an identity matrix 7 for B into this edquation, we will get the matrix 
inverse X = 4-17 = 4-1. We, however, usually use the MATLAB command 
inv(A) or A^-1 to compute the inverse of a matrix A. 


2.4 DECOMPOSITION (FACTORIZATION) 


2.4.1 LU Decomposition (Factorizatiom): Triangularization 


LU decomposition (factorization) of a nonsingular (Square) matrix 4 means 
expressing the matrix as the multiplication of a lower triangular matrix 乙 and 
an Upper triangular matrix UV，where a lowerupper triangular matrix is a matriX 
having no nonzero elements above/below the diagonal. For the case where Some 
IOwW Switching operation is needed like in the Gauss elimination，we include a 
permutation matrix 已 representing the necesSary IOw Switching operation(S) to 
write the LU decomposition as 


P4= 工 T (2.4.1) 
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The usage of a permutation matrix ls exemplified by 


0 0 1 QI1 012 013 031 032 033 
8 一 1 0 0 C21 C22 C23 一 C11 CQ12 C13 (2.4.2) 
0 1 0 031 4032 033 021 022 023 


which denotes Switching the first and third rows followed by Switching the second 
and third rowSs. An interesting and useful property of the permutation matrix is 
that lts transpose agrees with its inverse. 


P7P 一 1 P7 = Pr-i (2.4.3) 


To take a close look at the LU decomposition，we consider a 3 x 3 nonsingular 
ImatriX: 


QI al12 013 1 0 0 WUI1 MI12 113 

1 ad2 0d3|=|P2 1 0 0 122 123 

031 9032 033 61 5032 1 0 0 0133 

QI1 0412 9413 &11 以 12 &13 

01 0 03 | 三 | Pi PP1U12 十 2z22 121713 十 U23 (2.4.4) 
031 032 033 131U11 131U12 十 132U22 。 13113 十 132123 十 WU33 


First equating the first rows of both sides yields 
2U1n 一 Qln， 刀 .一 1,2,3 (2.4.9a) 
Then, equating the second rows of both sides yields 
021 三 PP1U11， 022 一 /21112 十 22， 023 一 121U13 十 U23 

from which we can get 

/21 一 021/x11， 122 一 021 一 121412， 123 一 023 一 121113 (2.4.5b) 
Now, equating the third rows of both Sides yields 

031 三 131U11， 032 三 131412 十 /32122， 033 一 01317413 十 U32123 十 U33 

from which we can get 


131 一 031/211， 132 一 (032 一 05317112)/z22， 1U33 一 (033 一 131U13) 一 /32123 
(2.4.5c) 
In order to put these formulas in one framework to generalize them for matri- 
ces having dimension greater than 3，we Split this procedure into two Steps 
and write the intermediate lowerupper triangular matrices into one matrix for 
compactness as 
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CI 412 913 
Step 起 CQ21  G22 0G23 
0431 032 033 


11 二 011 2&12 一 412 U13 二 413 
() () 
一 | PP = 0 三 022 一 121U12 023 三 023 一 121413 (2.4.0a) 
__ () (7 
131 三 031/UI1 03) 三 032 一 3112 033 二 033 一 131113 


&11 012 213 


Step 2: 一 121 U22 一 0 包 1M23 一 0 多 (2.4.0b) 


(D O) (0 
131 132 三 03? /ul22 033 一 033 一 132123 


This leads to an LU decomposition algorithm generalized for an N4 x N4 
nonsingular matrix as described in the following box. The MAILAB routine 
“1u_dcmp()” implements this algorithm to fnd not only the lowerupper 
triangular matrix 忆 and V7，but also the permutation matrix 已 . We run it for 
a3x3matrixto get 世 UV,and P and then reconstruct the matrix P-LLU=4 
from 忆 ,V, and P to ascertain whether the result is Tight. 


function [L,U,P] = Lu_ dcmp(A) 
%This gives LU decomposition of A With the permutation matrix P 
#% denoting the row Switch(exchange) during factorization 
NA = Size(A,1); 
AP = [A eye(NA)]; #%saugment With the permutation matrix . 
for k= 1:NA -1 

s%5Partial Pivoting at AP(k,k) 

[akx，kx]l = max(abs(AP(k:NA,k))); 

If akx < eps 

error('Singular matrix and No LU decomposition ) 
end 


Row change if necesSsary 
tmp_Fow AP(k,，:); 
AP(k，,:) AP(mx,:); 
AP(mx,:) = tmp_row; 
end 
%5 LU decomposition 
form= kr+1: NA 
AP(m,k) = AP(m,k)/AP(k,k); %Eq,(2.4.8.2) 
AP(m,k+1:NA) = AP(m,k + 1:NA)-AP(m,k)*AP(k,k + 1:NA); %Eq,(2.4.9) 
end 
end 
P=AP(1:NA，NAT+ 1:NA + NA); %Permutation matrIx 
for m = 1:NA 
for n = 1:NA 
ifm==n Lomni=1.; Unmnm = AP(mm); 
elseif my> n，Lmn) = AP(m;n); Unmn;n) = 0.; 
else L(m;n) = 0.; Unm;n) = AP(m;n); 
end 
end 
end 
if nargout == 0，disp('Lx*U = PxA With' ); LU,P，end 
%YOou can check if P'*LxU = A? 
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(cf) The number of foating-point multiplications required in this routine Lu_dcmp() is 








MA4-1 NM4-1 
> OM-DW-k+D= 》{M(M+D -CNM TD 二 
R 5 





一 (MA 一 UNMCNM 二 IT CNMA HTD)ONMA 一 TDNMAH+ 5 ])NMC2NM 一 了]) 


1 1 
一 了 CA 一 DJNM4A(CV4A 十 1] 六 了 NM4- (2.4.7) 


with NM4: the Size of matrix 4 


.JInitialize 440 = 4, or equivalently, al = am for 几 寻 一 1: NM4. 
. Let 一 二 站: 
一 0，do an appropriate ToOw Switching operation So that 


When it is not possible, then declare the case of Singularity and stop. 

: al 一 一 0 1) = xi ， for = 大 : N4 (ust leave the kth row as it is) 
(2.4.8a) 
0 由 =a 人 Da 人 = 天， form= 大 二 1:NA 0.4.8b) 
的 0 for mr, 一 大 十 1: NA (2.4.9) 


.IncrementKby 1 and 直上 < NA4 一 1 goto step 1; otherwise, go to Step 0. 


. Set the part of the matrix 4Ww4-D below the diagonal to 元 (lower tri- 
angular matrix with the diagonal of 1 s) and the part on and above the 
diagonal to V (upper triangular matrix). 





>>A= [125;0.21.67.4; 0.5 48.5]; 


>>[L,U;,P] = 1Lu_ dcmp(A) %LU decomposition 
L=1.0 0 0 U= 1 2 5 P=1 0 0 
0.5 1.0 0 0 3 6 0 0 1 
0.2 0.4 1.0 0 0 4 0 | 0 
>>P'*LxU - A %check the validity of the result (P' = P^-1) 
ans = 0 0 0 
0 0 0 
0 0 0 
>>[L,U,P] = LuU(A) %for comparison With the MATLAB built-in function 


What is the LU decomposition for? It can be used for solving a System of 
linear equations as 


4x 一 b (2.4.10) 


Once we have the LU decomposition of the coefficient matrix 4 王 P7ZU, it is 
more efficient to use the lowerupper triangular matrices for Solving Eq. (2.4.10) 
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than to apply the Gauss elimnination method. The procedure ls as follows: 


PT7LUX=-b ， LIUx=Pb Ux=LI Pb x=U ZI Pb 
(2.4.11) 
Note that the premultiplication of 苑 -1 and U-1 by a vector can be per- 
formed by the forward and backward substitution，respectively. The following 
program “do_lu_dcmp.m”applies the LU decomposition method，the Gauss 
elimination algorithm，and the MAILAB operators 八 ” and "inv” or 人 人 -1 to 
Solve Eq. (2.4.10)，where 4 is the five-dimensional Hilbert matrix (introduced 
in Example 2.3) andb = 4xo with x2 一 [1 1 1 1 1]7.Theresidual error 
4x 一 bll of the solutions obtained by the four methods and the numbers of 
foating-point operations Tequired for carrying out them are listed in Table 2.1. 
The table shows that once the inverse matrix 4-L is available, the inverse matrix 
method requiring only NM2 multiplications/additions (NV is the dimension of the 
coefficient matrix or the number of unknown variables) ls the most efftcient in 
computation，but the worst in accuracy. Therefore, if we need to continually 
Solve the System of linear equations with the Same coefficient matrix 4 for dif- 
ferent RHS vectors, it 18 a Teasonable choice in terms of computation time and 
accuracy to Save the LU decomposition of the coefficient matrix 4 and apply the 
forward/backward Substitution ProcesS. 


do_JlLu_dcmp 
Use LU decomposition，Gauss elimination to SoOLVe Ax 
= hilb(5); 
L,U,P] = Lu_dcmp(A); #%LU decomposition 
= [1-23.-45-67 -89 -10] 
b = A*x(1:Size(A,1)); 
flops(0)，x_JlLu = backsubst(U,forsubst(L,Px*b)); %5Eq.(2.4.11) 
flps(1) = flops; s% assuming that we have already got LNU decomposition 
flops(0) ，Xx_gs gauss(A,b); flps(3) = flops; 
flops(0) ，Xx_bs A\b; flps(4) = flops; 
AI = A^-1; flops(0)，x_iv = AI*b; flps(5) = flops; 
5 asSuming that we have already got the inverse matriX 
disp(” X_1U X_gs X_bs X_iv ' ) 
format Short e 
Solutions = [Xx_lu x gs xbs x_ivl] 
errs = [norm(A*x_lIuU - b) norm(A*x gs - b) norm(A*x_bs - b) norm(Ax*x_iv - b)] 
format Short，f1lps 


function x = forsubst(L,B) 
s%5forward Substitution for a lower-triangular matrix equation LXx = B 
N = Size(L,1); 
Xx(1,:) = B(1，:)/ 人 (1 1) 
form=2:N 
Xx(m,:) = (B(m,:)-Lm;1:m- 1)*x(1:m-1，:))/Lmm); 


function x = backsubst(U,B) 
s%5backward Substitution for a upper-triangular matrix equation Ux = B 
Size(U,2); 





(mm+ 1:N)xxm+t 1:N:))/UCmm) 
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Table 2.1 Residual Error and the Number of Floating-Point Operations of Various 
Solutions 


tmp = forsubst(L,Pxb) 


backsubst(U,tmp) gauss(A,b) A\b A^ 人 -1*b 
4x 一 bl|| 1.3397e-010 $.5911e-017 1.73$4e-010 3.093Se-012 
# of fops 123 224 155 50 


(cf The numbers of fops for the LU decomposition and the inverse of the matrix 4 are not counted. 


(cf) Note that the command “fops” to count the number of foating-point operations is no longer 
available in MAILAB 6.x and higher versions. 


2.4.2 Other Decomposition (Factorization): Cholesky, QR, and SVD 


There are Several other matrix decompositions Such as Cholesky decomposition， 
QR decomposition, and Singular value decomposition (SVD). Instead of looking 
into the details of these algorithms,， we will simply survey the MAILAB built-in 
functions Implementing these decompositions. 

Cholesky decomposition factors a positive definite symmetric/Hermitian matriX 
into an upper triangular matrix premultiplied by its transpose as 


4=V7IU (LV: an upper triangular matrix) (2.4.12) 
and is Implemented by the MAILAB built-in function chol(). 


(cf) If a (complex-valued) matrix 4 satisfies 4*7 一 4 一 that is, the conjugate transpose 
of a matrix equals itself 一 it is Said to be Hermitian. It is Said to be just Symmetric 
in the case of a real-valued matrix with 47 = 4. 


(cp If a square matrix 4 satisfies xx74X> 0YX 天 0,the matrix is said to be positive 
definite (See Appendix 了 B). 


>>A [234;356;4.69]; %a positive definite Symmetric matrIixX 
>>U = chol(A) #%Cholesky decomposition 
U = 1.4142 2.1213 2.8284 
0 0.7071 0.0000 
0 0 1.0000 
>>U'*U - As%to check if the result is right 


QR. decomposition is to express a Square or rectangular matrix as the product 
of an orthogonal (unitary) matrix CO and an upper triangular matrix 尺 as 


4=OR (2.4.13) 


where 0O70 =7T (070 = 门 .This is implemented by the MAILAB built-in 
function qr(). 
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(ch) 开 allthe columns of a (complex-valued) matrix 4 are orthonormal to each other 一 that 
is, 4 4 = 1, or, equivalently, 4x7 = 4-1 一 itis said to be unitary. It is said to be 
orthogonal in the case of real-valued matrix with 47 一 4-!. 


SVD (Singular value decomposition) ls to express an M x N matrx 4 in the 
following form 
4=VUSVT7 (2.4.14) 


whpere UV is an orthogonal (unitary) M x M matrix，V is an orthogonal (uni- 
tary) NM x N matrix, and 9 is a real diagonal M x N matrix having the Sin- 
gular values of 4 (the square roots of the eigenvalues of 47 4) in decreasing 
order on its diagonal. This is Implemented by the MAILAB built-in function 
svd(). 


>>A= [1 2;2 3;3 5]; %a rectangular matrIix 
>>[U,S,V] = Svd(A) %5Singular Value Decomposition 
U = 0.3092 0.7557 -0.5774 SS= 7.2071 0 VvV = 0.5184 -0.8552 
0.4998 -0.6456 -0.5774 0 0.2403 0.8552 0.5184 
0.8090 0.1100 0.5774 0 0 
>>err = Ux*Sx*V' -A %to check if the result is _ right 
err = 1.0e-015* -0.2220 -0.2220 
0 0 
0.4441 0 


2.5 ITERATIVE METHODS TO SOLVE EQUATIONS 


2.5.1 Jacobi lteration 
Let us consider the equation 

3x 十 1 一 0 
which can be cast into an iterative Scheme as 


X 十 1 1 1 
二 E > 证 本 ] 
大 大 ; 苇 KK 十 1 7 7 








Starting from some initial value xo forK = 0, we can incrementally change 大 
by 1 each time to proceed as follows: 





X1 一 一 2-1 一 2-1x0 
X 一 一 2-1 一 2-1xl 一 一 2 十 2 一 十 2-2x0 
1 


Whatever the initial value xo 1$，this process will converge to the Sum of a 
geometric series with the ratio of (一 1/2) as 
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a0 一 1/2 1 
XU 二 一 一 一 一 一 一 一 一 X as K 一 CO 
1 一 站 一 人 OO 3 


and what is better，the limit is the very true Solution to the given equation， 
We are happy with this，but misght feel uneasy，because we are afraid that this 
Convergence to the true solution ls just a coinclidence. Will it always converge， 
no matter how we modify the equation So that only x remains on the LHS? 

To answer this question, let us try another iterative Scheme. 


大 一 一 27 一 1 一 Xi+Hl 王 一 2 太一 1 
一 一 一 2xX0 
xi 一 -1 一 2x1 二 -1 一 2(-1 一 2xo) 二 一 1 十 2 十 22x0 





| 
这 
下 
iD 
iD 

Le 





X3 一 一 1 一 2x2 一 23x0 


This iteration will diverge regardless of the initial value xo. But，we are never 
disappointed, Since we know that no one can be always lucky. 

To understand the essential difference between these two cases，we Should 
know the fxed-point theorem (Section 4.1). Apart from this, lef rs go into a System 


of equations. 
3 -多 Xl1 | 1 人 
和 


Dividing the first equation by 3 and transposing all term(S) other than xl to the 
RHS and dividing the second equation by 2 and transposing all term(S) other 
than x2 to the RHS，we have 


下 
X2 .HH1 于/ X2 居 一 1/2 
Pb 


xel 一 4 区 十 





(2.5.1) 


Assuming that this Scheme works well， we set the initial value to zero (xo 一 0) 
and proceed as 


1/2 1 一 1/2 


呈 1 1 一 2/3 l/3 | 1 工 |23 1 下 汉 xo 
1=-1/3|1-L2 1 -1/2| 2/31-23| -可 
(5 . 坊 
which will converge to the true solution x2 = [1 一 1] This suggests another 


method of solving a System of equations，which is called Jacobi iteration. It can 
be generalized for an NM x N matrix-vector equation as follows: 


一 1 
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ClX1 十 Or2X2 十 十 GomrXom 十 十 GorNXN 一 Dm 


N 
1177 Do 
0 
九天 11 CGI17 Cjzz71 
Xk+1 一 人 XK 十 b for each time Stage 大 (2.5.3) 
where 
| 0 一 012/a11 EN D1/all 
一 一 0p1/a 0 一 apDN/aG 僵 Do /a 
和 21/ 22 2 2N/ 2 | = 2/ 22 
一 NI/GNN 一 GN2/GNN  …: 0 DN/aNN 


3? 


This Scheme is Implemented by the following MAILAB routine“jacobi() 7”. 
We run it to Solve the above equation, 


function X= jacobi(A,B,X0o,kmax) 
%This function finds a Soltuion to AX = B by Jacobi iteration. 
if nargin < 4，tol = 1e-6; kmax = 100; %called by jacobi(A,B,X0) 
elseif kmax < 1，tol = max(kmax,1e-16); kmax = 100; 5%jacobi(A,B,X0,tol) 
else tol = 1e-6; 5%5jacobi(A,B,X0,kmax) 
end 
if nargin < 3，X0 = zeros(Size(B)); end 
NA = Size(A,1); 
X = X0j At = zeros(NA,NA); 
for m = 1:NA 
for n = 1:NA 
ifn=-= mAtmn) = -Am;n)/VACmm); end 
end 
Bt(m,:) = B(m,:)/ACmm); 
end 
for k = 1: kmax 
X = At*X + Bt;j %Eq， (2.5.3) 
if nargout == 0，X，end s%5To see the intermediate results 
if norm(X - X0)/(norm(X0) + eps) < tol，break;i end 
XO0 = Xi; 
end 





>>A = [32;12];b= [1 -1]' ; %the coefficient matrix and RHS vector 
>>Xx0 = [0 0] ' ; %the initial value 
>>X = jacobi(A,b,x0,20) %to repeat 20 iterations Starting from X0 
Xx = 1.0000 
-1.0000 
>>]jacobi(A,b,x0,20) %omit output argument to see intermediate results 
X = 0.3333 0.6667 0.7778 0.8889 05-9259 aas 
-0.5000 -0.6667 -0.8333 -0.8889 -0.9444  .，,.,,， 


2.5.2 Gauss-Seidel Iteration 


Let us take a close look at Eq. (2.3.1). Each iteration of Jacobi method updates 
the whole set of N_ variables at a time. However，So long as we do not use a 
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multiprocesSor computer capable of parallel processing, each one of N variables 
is updated sequentially one by one. Therefore, it ls no wonder that we could 
Speed up the convergence by using all the most recent values of variables for 
updating each variable even in the Same iteration as follows: 


2 1 
X1,k+H1 一 一 了 2 十 3 

1 1 
X2, 人 二 1 一 一 ULK+1 二 7 





This Scheme is called Gauss-Seidel iteration，which can be generalized for an 
N x N matrix-vector equation as follows: 


yl +HD SN (0) 
大 (+1) Do 7 Ca 2 Ca 
人 
Cj1z71z 


for 2 二 1...,Nandtfor each time Stage 大 (2.5.4) 


This ls implemented in the following MAILAB routine “gauseid()”， which 
we will use to Solve the above equation. 


function X = gauseid(A,B,X0,kmax) 
%This function finds x = A^-1 B by Gauss-Seidel iteration. 
if nargin < 4，tol = 1e-6;j kmax = 100; 
elseif kmax < 1，tol = max(kmax,1e-16); kmax = 1000; 
else tol = 1e-6; 
end if nargin < 4，tol = 1e-6j kmax = 100; end 
if nargin < 3，X0 = zeros(Size(B)); end 
NA = Size(A,1); X = X0; 
for k = 1: kmax 
X(1,:) = (B(1,:)-A(1,2:NA)*X(2:NA,:))/A(1,1) 
for m= 2:NA-1 
tmp = B(m,:)-A(m;1:m-1)*Xx1:m- 1，:)-AGmm+ 1:NA)*Xm + 1:NA,:); 
X(m,:) = tmp/A(m,m); %5Eq.(2.5.4) 
end 
X(NA,:) = (B(NA,:)-A(NA,1:NA - 1)*X(1:NA - 1，:))/VA(NA;NA) | 
if nargout == 0，X，end 5%5To See the intermediate results 
if norm(X - X0)/(norm(X0) + eps)<tol，break;i end 
XO0 = Xi 





>>A = [32;j12];b= [1 -1] ; %the coefficient matrix and RHS Vector 
>>Xx0 = [0 0] ' ; %the initial Value 
>>gauseid(A,b,Xx0,10) s%5omit output argument to See intermediate results 
X = 0.3333 0.7778 0.9259 0.9753 059918:… -ec 
-0.6667 -0.8889 -0.9630 -0.9877 -0.9959  .,,,,， 


As with the Jacobi iteration in the previous Section, we can See this Gauss-9eidel 
iteration converging to the true solution x2 = [1 一 17 and that with fewer iter- 
ations. But, 让 we use a multiprocessor computer capable of parallel processing， 
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the Jacobi iteration may be better in Speed even with more iterations, Since it can 
exploit the advantage of Simultaneous parallel computation. 

Note that the Jacobl/Gauss-9Seidel iterative Scheme Seems unattractive and 
even Unreasonable if we are given a standard form of linear equations as 


4X 一 b 


because the computational overhead for converting it into the form of Eq. (2.5.3) 
may be excesSlve. But, it ls not always the case, esSpecially when the equations 
are given in the form of Eq. (2.3.3)/(2.3.4). In Such a case，we Simply Tepeat 
the iterations without having to use Such ready-made routines as“jacobi()”or 
“gauseid()”. Let us see the following example. 


了 kxXample 2.4. Jacobi or Gauss-Seidel Iterative Scheme. Suppose the tempera- 
ture of a metal rod of length 10 m has been measured to be 0"C and 10"C at 
each end, respectively. Find the temperatures xl, X2, X3, and xX4 at the four points 
equally spaced with the interval of 2 m, assuming that the temperature at each 
point is the average of the temperatures of both nelighboring points. 

We can formulate this problem into a System of equations as 


X0 十 X2 X1 十 X3 X2 十 X4 
X1 二 二 X2 ss 0 3 一 一 
2 2 2 
X3 十 X5 
充 权 二 0 with xo 一 0 and x5 一 10 (上 2.4) 


This can easily be cast into Eq. (2.3.3) Or Eq. (2.3.4) as programmed in the 
following program“nm2e04.m: 


%nm2e04 

N = 4; %the number of unknown variables/edquations 

kmax = 20; tol = 1e-6; 

At= [0100;1010;0101; 0010]/2; 

x0 = 0; x5 = 10; %boundary values 

b = [x0/2 0 0 x5/2] ' ; %5RHS Vector 

%initialize al1 the values to the average of boundary values 

Xp=ones(N,1)*(x0O + Xx5)/12; 

%Jacobi iteration 

for k = 1:Kkmax 
X = AtxXxp +b;j %5Eq.(E2.4) 
if norm(x - xp)/(norm(xp)+eps) < tol，break;i end 
Xp = Xi 

end 

Kk，Xj = X 

%5Gauss-Seidel iteration 

xp = ones(N,1)*(x0 + X5)/2; X = Xpj %initial Value 

for k = 1:Kkmax 
forn= 1:N，Xxn) = Atln;，:)*x+ bln); end %5Eq.(E2.4) 
if norm(x - xp)/(norm(xp) + eps) < tol，breakji end 
Xp = Xi 

end 

Kk，Xxg = X 
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The following example jllustrates that the Jacobi iteration and the GausS-9eidel 
iteration can alsobeusedfor solving asystem ofnonlinear equations, althoughthere 
is no guarantee that it will work for every nonlinear equation 


了 Example 2.S. Gauss-9Seidel Iteration for Solving a Set of Nonlinear Equations， 
We are going to use the Gauss-Selidel iteration to Solve a System of nonlinear 
equations as 
好 十 10x1 十 2x2 一 13 一 0 
(上 E2.3.1) 
2x 一 好 二 Sa 一 6=0 


In order to do so,，we convert these equations into the following form，which 
Suits the GausS-Seidel Scheme. 


x1| | (413 一 并 一 2x2)/10 
加 有 | (6 一 2x 十 好 )/5 (E2.5.2) 


We make the MAILAB program“nm2e05.m"，which uses the GausS-Seidel 
iteration to Solve these equations. Interested readers are recommended to run 
thls program to see that this Simple iteration yields the solution within the given 
tolerance of error in just Six StepS. How marvelous jlt ls to Solve the System of 
nonlinear equations without any Special algorithml! 


(cf) Due to its remarkable capability to deal with a system of nonlinear equations, the 
GausS-Seidel iterative method plays an important role in Solving partial differential 
equations (See Chapter 9). 


%nm2e05 .nm 
s%5 USe Gauss-Seidel iteration to Solve a Set of nonlinear equations 
Clear 
kmax = 100; tol = 1e-6; 
Xx = zeros(2,1); %initial Value 
for k = 1:Kmax 
xp = Xj 5% to _ remember the previous Solution 
(1 (13 - x(1)^2 - 2*x(2)^2)/110; 5 (E2.5.2) 
(2 (6 - X(1)^3)715; 
if norm(x - xp)/(norm(xp) + eps)<tol，break;i end 
end 
K ， X 


久 
义 


) 
) 





2.5.3 The Convergence of Jacobi and Gauss-Seidel lterations 


Jacobli and Gauss-Seidel iterations have a very Simple computational Structure 
because they do not need any matrix inverslion. So, it may be of Practical use, 让 
only the convergence is guaranteed. However, everything cannot always be fine， 
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as jllustrated in Section 2.3.1. Then，what ls the convergence condition? It is the 
diagonal dominancy of coefficient matrix 4，which is stated as follows: 


NV 
loan| > 》， la ， form 二 12...，N (2.5.5) 
用 天 11 


This implies that the convergence of the iterative Schemes is ensured 1 寺 ，in 
each Tow of coefficient matrix 4, the absolute value of the diagonal element 
js greater than the sum of the absolute values of the other elements. It should 
be noted, however, that this is a sufficient, not a necessary，condition. In other 
words, the iterative Scheme may work even if the above condition is not Strictly 
Satisfied. 

One thing to note is the relaxation technique，which may be helpful in accel- 
erating the convergence of Gauss-Seidel iteration. It is a slight modifcation of 
了 Eq. (2.3.4) as 


凡 一 1 (KK 十]) 和 (人 
Do 2 Com 人 六 力 二 河 Cam 


Xe ) 一 (] 一 CO)XW) 十 O 
Cj1z71 


with0<w<2 (2.3.0) 


and is called SOR (Successive overrelaxation) for the relaxation factor 1 < w 去 
2 and SucceSsive underrelaxation for 0 < w < 1. But regrettably,， there is no 
general rule for selecting the optimal value of the relaxation factor ww, 


PROBLEMS 


2.1 Recursive Least-9quares Estimation (了 RLSE) 
(a) Run the program “do_rlse.m (in Section 2.1.4) with another value of 
the true parameter 
xo = [1 2]， 


What is the parameter estimate obtained from the RLS solution? 
(b) Run the program“do_rlse” with a small matrix 忆 like 


P = 0.01xeye(NA) ; 


What is the parameter estimate obtained from the RLS solution? IS 
still close to the value of the true parameter'? 

(c) Insert the statements in the following box at appropriate places in the 
MAILAB code“do_rlse.m” appeared in Section 2.1.4. Remove the 
last two statements and run it to compare the times required for using 
the RLS solution and the Standard LS solution to get the parameter 
estimates on-line. 
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%nm2p01 . 


time_ on = 0; time_off 

七 IC 

time_on time_on + toc 

七 IC 

Xxk_off = AN\bj; %standard LS solution 
time_off = time_off + tocij 
SoOlutions = [x xk_ off] 

discrepancy = norm(x - Xk_off) 
times = [time_on time_off] 





2.2 Delicacy of Scaled Partial Pivoting 


As a_ complement to Example 2.2，we want to compare no pivoting，Ppar- 
tial plvoting， Scaled partial pivoting, and full pivoting in order to taste the 
delicacy of row Switching Strategy. To do it in a Systematic way,，add the 
third Input argument (pivoting) to the Gauss elimination routine “gauss()” 
and modify its contents by inserting the following statements into appropri- 
ate places So that the new Ioutine “gauss(A,b,pivoting)”implements the 
partial pivoting procedure optionally depending on the value of "pivoting . 
You can also remove any unnecesSary parts. 


- if nargin < 3，pivoting = 2; end %Sscaled partial pivoting by defau]t 
- Switch pivoting 
case 2，[akx,kx]l = max(abs(AB(k:NA,k))./.,. 
max(abs([AB(k:NA,k + 1:NA) epsx*oneSs(NA - k+ 1,1)] )) ); 
otherwise，[akx,kx]l = max(abs(AB(k:NA,k))); %partial pivoting 
end 
- &pivoting > 0 %partial pivoting not to be done for pivot = 1 
(a) Use this routine with pivoting = 0/11/2，the 八 ” operator and the 
“inv()”command to solve the systems of linear equations with the 
coefficient matrices and the RHS vectors Shown below and fll in 
Table P2.2 with the residual error ||4iX 一 bill to compare the results 
in terms of how well the solutions Satisfy the equation，that 1S， 


上 4 一 | ss 0. 
10-55 1 1 十 107-5 
加 4=| 1 | pn =| oa 
10-14.6 1 二 二 10-14.6 
四 =| 1 1 m=| 1025 十 1 | 


101 1 101 十 1 
风 =| 1 | =| 性 这 5] 
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Table P2.2 Comparison of gauss () with Different Pivoting Methods in Terms of 
11Axi -bll 


AIX 一 bi AD2X 一 b> A3X 一 bs3 4X 一 ba4 
gauss(A;,b,0) (no pivoting) 1.2S9e-01 
gauss(A;b,1) (partial pivoting) 4.44e-16 
gauss(A,b,2) (Scaled partial pivoting) 0 
A\b 6.29e-02 


A^-1x*b 


1014.6 1 1014.6 十 1 
人 4=| 1 和 用 二 | 


(b) Which pivoting Strategy yields the worst result for problem (1) in (a)? 
Has the row Swapping been done during the process of partial pivoting 
and scaled partial pivoting? If yes, did it work to our advantage7 Did 
the 人 operator or the “inv()”command give you any better result? 


(c) Which pivoting Strategy yields the worst result for problem (2) in (a)7 
Has the row Swapping been done during the process of partial pivoting 
and scaled partial pivoting? If yes, did it produce a positive effect for 
thls case7 Did the 信 ” operator or the “inv()”command give you any 
better result? 


(d 


S 


Which pivoting Strategy yields the best result for problem (3) in (a)7 Has 
the row Swapping been done during the process of partial pivoting and 
Scaled partial pivoting? HI yes, did it produce a positive effect for this 
Case7 


(e 


旋 


The coefficient matrix 43 ls the Same as would be obtained by applying 
the full pivoting scheme for 4l to have the largest pivot element. Does 
the full pivoting give better result than no pivoting or the (Scaled) partial 
pivoting” 

( Which plivoting Strategy yields the best result for problem (4) in (a)7 Has 
the row Swapping been done during the process of partial Pivoting and 
Scaled partial pivoting? HI yes, did it produce a positive effect for this 
case? Did the 人 ”operator or the “inv()”command give you any better 
TeSsult2 


2.3 GausS-Jordan Elimination Algorithm Versus Gauss Elimination Algorithm 


GausS-Jordan ellimination algorithm mentioned in Section 2.2.3 ls trimming 
the _ coefficient matrix 4 into an identity matrix and then takes the RHS 
vectormatrix as the Solution，while Gauss elimination algorithm introduced 
with the corresponding routine “gauss()”in Section 2.2.1 makes the matrix 
an upper-triangular one and performs backward Substitution to get the Solu- 
tion. Since Gauss-Jordan elimination algorithm does not need backward 
Substitution, it Seems to be Simpler than Gauss elimination algorithm. 


PROBLEMS 107 


Table P2.3 Comparison of Several Methods for Solving a Set of Linear Equations 


gausSs(A,b) gaussj(A,b) A\b 人 A-1*b 
4x; 一 bl 3.1402e-010 8.7419e-0106 


# of fops 1124 1744 785 7670 


(a) Modify the routine “gauss()”into a routine“gaussj()”which imple- 
ments Gauss-Jordan elimination algorithm and count the number of 
multiplications consumed by the routine，excluding those required for 
partial pivoting. Compare it with the number of multiplications consumed 
by“gauss()”[Eq. (2.2.18)]. Does it Support Or betray OUIT expecta- 
tion that Gauss-Jordan elimination would take fewer computations than 
Gauss eliminationy 

(b) Use both of the routines, the 人 ”operator and the “inv()”command or 
-1 to solve the System of linear equations 


4x 一 hb (P2.3.1) 


where 4 is the 10-dimensional Hilbert matrix (See Example 2.3) and 
b=4xo withxo=[111111111H7 FinTableP23withthe 
residual errors 

14x 一 bll 关 0 (P2.3.2) 


as a Way of describing how well each solution Satisfies the equation. 


(cf) The numbers of foating-point operations required for carrying out the 
computations are listed in Iable P2.3 so that readers can compare the com- 
putational loads of different approaches. Those data were obtained by using 
the MAILAB command flops()，which is available only in MAILAB of 
version below 0.0. 


2.4 Tridiagonal System of Linear Equations 


Consider the following System of linear equations: 








QIIX1I 十 G127X2 一 DO 
C21X1 十 0227X2 十 023X3 一 22 
本 (P2.4.1) 
QN_1LN_2XN_2 十 GN_1LN_IXN_1 十 GN_LNXN 一 DN-l 
QN,N-IXN-1 十 GN,NXN 一 DN 





which can be written in a compact form by using a matrix-vector notation as 


ANxNX 一 b (P2.4.2) 
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Table P2.4 The Computational Load of the Methods to Solve a Tri-diagonal 
System of Equations 





gausSs(A,b) trid(A,b) gauseid () gauseid1() A\b 
## of fops 141 50 2015 2082 94 
Where 
CGI1 G12 0 0 0 
021 022 023 0 0 
4Nwxvw 三 | 0 0 ， 
0 0 alLN-2 QN_LN_1 QN_LN 
0 0 0 CQGN,N-1 CNN 
X1 DPI 
X2 2D2 
:一 Di 三 
XN_1 [ 
XN DN 


This is called a tridiagonal System of equations on account of that the 
coefficient matrix 4 has nonzero elements only on its main diagonal and 
Super-/Subdiagonals. 


(3) 


(b) 


(C) 


Modify the Gauss ellimination routine “gauss()”(Section 2.2.1) in Such 
a way that this Special Structure can be exploited for reducing the com- 
putational burden. Give the name "trid() ”to the modified routine and 
Save lt in an m-fle named “trid.m” for future use. 


Modify the Gauss-Seidel iteration Toutine “gauseid()”(Section 2.3.2) 
in Such a way that this Special Structure can be exploited for reduc- 
ing the computational burden. Let the name of the modified routine be 
“Gauseid1()”. 


Noting that Eq. (E2.4) in Example 2.4 can be trimmed into a tridiago- 
nal Structure as (P2.4.2),，use the routines“gauss()” “trid()” “gau- 


seid()”,“gauseid1()”，and the backslash (\) operator to solve the 

problem. 

(cf) The numbers of foating-point operations required for carrying out the 
computations are listed in Table P2.4 so that readers can compare the com- 
putational loads of the different approaches. 


LU Decomposition of a Tridiagonal Matrix 


Modify the LU decomposition routine “lu_dcmp()”(Section 2.4.1) in such a 
way that the tridiagonal structure can be explolted for reducing the 
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computational burden. Give the name“1lu_trid()”to the modiftied routine 
and use it to get the LU decomposition of the tridiagonal matriX 


4 一 (P2.5.1) 


100; -12 -10;0-12 -1;00 -1 2]; 
= lu trid(A) 
>>LxrU - A5%55=0 (No error)? 


LS Solution by Backslash Operator and QR Decomposition 


The backslash (ANb”) operator and the matrx left division 
(mldivide(A,b)”) function turn out to be the most efficient means for solv- 
ing a System of linear equations as Eq. (P2.3.1). They are also capable of 
dealing with the underover-determined cases. Let's See how they handle the 
Underover-determined cases. 


(a) For an underdetermined System of linear equations 


1 

2 14 

AI1X 一 bi ， 攻 5 X2 | 一 的 (P2.0.1) 
X3 


fnd the minimum-norm solution (2.1.7) and the solutions that can be 
obtained by typing the following statements in the MAILAB command 
WwWindow: 


>>A1= [123;j456]; bl1= [14 32] ; 
>>Xx_mn = A1'*(A1*A1 )^-1*b1，x_pi = pinv(A1)*b1，x_bs = Al\bl 


Are the three solutions the same? 


(b) For another underdetermined System of linear equations 
好 1 
2 14 
3 


fnd the solutions by using Eq. (2.1.7), the commands pinv(), and back- 
slash (\). 玉 you are not pleased with the result obtained from Eq. (2.1.7)， 
you can remove one of the two rows from the coefficient matrix 4 and 
try again. Identify the minimum solution(S). Are the equations redundant 
Or inconsistent7 
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(d 


(e 


) 


) 
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Table P2.6.1 Comparison of Several Methods for Computing the LS Solution 


QR LS: Eq. (2.1.10) pinv(A)*b A\b 
|4x 一 bll 2.8788e-016 2.8788e-016 
# of fops 25 89 190 92 


FEor another underdetermined System of linear equations 
X1 
1 2 3 21 
3 


fnd the solutions by using Eq. (2.1.7), the commands pinv(), and backc- 
Slash (\). Does any of them satisfy Eq. (P2.0.3) closely? Are the equations 
Tedundant or inconsistent?7 


For an overdetermined System of linear equations 


1 2 5.2 
44X 一 ba， 2 3 攻 | 一 | 7.8 (P2.6.4) 
4 2.2 


fnd the LS (east-squares) solution (2.1.10), that can be obtained from 
the following statements. Fill in the corresponding blanks of Iable P2.6.1 
with the results. 


>>A4 = [12;j23j4 -1]; b4= [5.27.82.2] ; 
>> X_ 1S = (A4'*A4)\A4'*b4，Xx_pi = pinv(A4)*b4，Xx_bs = A4\b4 


We can use QR decomposition to solve a System of linear equations as 
Eq. (P2.3.1), where the coefficient matrix 4 is Square and nonsingular or 
rectangular with the row dimension greater than the column dimenslion. 
The procedure ls explained as follows: 


4x= ORx 一 b， Rx=O-ihb= 0Om， x 一 R-LIOb (P2.6.5) 


Note that OO0 = 7 0' = 0O-1 (orthogonality) and the premultiplica- 

tion of R-1L can be performed by backward substitution，because R is 

an upper-triangular matrix， You are Supposed not to count the num- 

ber of foating-point operations needed for obtaining the LU and QR 

decompositions, assuming that they are available. 

(D Apply the QR decomposition, the LU decomposition，Gauss elimi- 
nation,， and the backslash (\) operator to Solve the System of linear 
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Table P2.6.2 Comparison of Several Methods for Solving a System of Linear 


Equations 

LU QR. gauss(A,b) A\b 
4x 一 bl|| 7.85$0Se-0106 8.7419e-010 
# of fops 453 327 1124 785 


equations whose coefficient matrix is the 10-dimensional Hilbert 
matrix (See Example 2.3) and fll in the corresponding blanks of 
Taple P2.6.2 with the results. 


(i Apply the QR. decomposition to Solve the System of linear equations 
given by Eq.，(P2.6.4) and fl in the corresponding blanks of 
Taple P2.6.2 with the results. 

(cf) This problem illustrates that QR _ decomposition is qujite usefu]l for solving 
a System of linear equations，where the coeffticient matrix 4 is square and 
nonsingular or rectangular with the row dimension greater than the column 
dimension and no rank deficiency. 


2.7 Cholesky Factorization of a Symmetric Positive Definite Matrix: 


Ha matrix 4 1 Symmetric and positive definite，we can find its LU 

decomposition Such that the upper triangular matrix UV is the transpose of 

the lower triangular matrix 也，which is called Cholesky factorization. 
Consider the Cholesky factorization procedure for a 4 x 4 matriX 








al 012 ad13 014 MU 0 0 0 WUI1 MUI2 MI3 MU14 
a2 ad2 03 0d4| | zz 0 0 0 zx22 123 124 
a3 023 ad33 604 | 123 za xz 0 0 0 133 134 
QI14 024 034 044 1I14 124 134 144 0 0 0 1H4 
2 U1I1U12 11U13 MI1I1U14 
| 12V11 2 生 22 WU12113 十 U22123 WU1I2U14 十 U22U24 
| aa za3U12 十 U23122 1 十 2 于 人 33 U13U14 十 WU23U24 十 U33134 
WU14U11 MI4U12 十 U24122 MI14U13 十 WU24123 十 34133 04 中 1 本 3 机 2 和 





2 大 二 
Equating every Iow of the matrices on both Slides yields 


U11 三 Vall， 1U12 三 012/UI1，U13 一 013/U11，V14 三 0G14/U11  (P2.7.2.1) 


122 一 /ad22 一 0，， 1U23 一 (023 一 UI3U12)/U22，UM24 一 (024 一 ULI4U12)/U22 


(P2.7.2.2) 


2 2 
1U33 一 V da33 一 023 一 013， 1U34 一 (043 一 24123 一 U14113)/U33 (P2.7.2.3) 


2U44 一 /da44 一 134 一 124 一 14 (P2.7.2.4) 
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which can be combined into two formulas as 


天 一 1 
2 一 /ci 一 有 2 forK 一 1: N (P2.7.3a) 


Zn 一 (mw 生 三 na /ui form = 一 大 二 1:Nand 大 =1:N 
(P2.7.3b) 
(a) Make a MAILAB routine“cholesky()”, which Implements these for- 
mulas to perform Cholesky factorization. 
(b) Try your routine“cholesky()”for the following matrix and check 下 
7IU7 -4sO(V:the uppertriangular matrix). Compare the result with 
that obtained by using the MAILAB built-in routine“chol()”. 


1 2 4 由 

2 13 23 38 
4=|4 23 77 122 (人 274) 
7 38 122 294 


(c) Use the routine“lu_ dcmp()”and the MATLAB built-in routine “lu()” 
to get the LU decomposition for the above matrix (P2.7.4) and check 下 
PIT7LU -4%S0O, where 工 and Varethe lowerupper triangular matrix， 
IesSpectively. Compare the result with that obtained by using the MAT- 
LAB built-in routine“1lu()”. 


Usage of SVD (Singular Value Decomposition) 


What is SVD good for?” Suppose we have the Singular value decomposition 
of an M x N real-valued matrix 4 as 


4=VUSV7 (P2.8.1) 


where UV is an orthogonal M x M matrix, V an orthogonal N x N matrix， 
and 9 areal diagonal M x N matrix having the Singular value os of 4 (the 
Square roots of the eigenvalues of 47 4) in decreasing order on its diagonal. 
Then，it ls possible to improvise the pseudo-inverse even in the case of 
rank-deficient matrices (with Tank(4) < min(Af, NW)) for which the lefyright 
pseudo-inyverse can't be found. The virtual pseudo-inverse can be written as 


4 一 六 7 (P2.8.2) 


where Sr-1is the diagonal matrix having 1/o on its diagonal that is Tecon- 
structed by removing all-zero(-like) rows/columns of the matrix 9 and Substi- 
tuting 1/oi for o 和 关 0 into the resulting matrix; 乡 and 忌 are reconstructed 
by removing the columns of V and V corresponding to the zero Singular 
value(S). Consequently，SVD has a Specialty in dealing with the Singular 
caseSs. Let us take a closer look at this through the following problems. 
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(a) Consider the problem of solving 


X1 
业 二 3 0 
3 


Since this belongs to the underdetermined case (M =2<3 王 N), It 
Seemas that we can use Eq. (2.1.7) to find the minimum-norm solution. 


名 


二 


Type the following statements into the MAILAB command window. 
>>A1= [123;246];bl= [6;12]; x= Al'*(AlxA1') -1*bl %Eq， (2.1.7) 
What is the result? Explain why it is so and Support your answer by 
typing 

>>F = rank(A1 ) 

Type the following statements into the MAILAB command window 


to See the SYD-based minimum-norm solution. What ls the value of 
x 一 4TIbl=YS-IU7byand hx 一 bill? 


[U,S,V] = Svd(A1); 55(P2.8.1) 
U=U:r);jv=Vv::r);sSs=Sr1:r)i 
AIP = vx*diag(1./diag(Ss))*u'; s%faked pseudo-inverse (P2.8.2) 


Xx = AIpx*b1 %minimum-norm Solution for Singular underdetermined 
err = norm(A1*Xx - b1) %residual error 





Gi 


To see that the norm of this solution ls less than that of any other 
Solution which can be obtained by adding any vector in the null space 
of the coeffticient matrix 41, type the following statements into the 
MAILAB command window. What is Implied by the result? 


nulJA = nullL(A1); normx = norm(x) ; 
for n = 1:1000 
if norm(x + nullAx(rand(Size(nullA,2),1)-0.5)) < normx 


disp('What the hell1 smaller-norm Sol - not minimum norm' ) ; 
end 
end 





(b) For the problem 


区 1 
1 2 3 0 
3 


compare the minimum-norm Solution based on SVD and that obtained 
by Eq. (2.1.7). 
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(cC) Consider the problem of solving 


1 2 3 。 国 
4 5 9 4 2 
43 一 7 11 18 2 一 3 一 bs3 (P2.8.9) 
2 全 可 4 


Since this belongs to the overdetermined case (M =4>3=NW)，L 
Seemas that we can use Eq. (2.1.10) to find the LS (least-squares) Solution. 
(iD Type the following statements into the MAILAB command window: 


>>A3=[12 3;459;7 1118;-2 3 1]; 
>>b3=[1;2;3;4]; x=(A3'*A3)^-1x*A3'*b3 %Eq，(2.1.10) 


What is the result? Explain why it ls So in connection with the Tank 
of 43. 


(让 Similarly to (a)Gi), fnd the SVD-based leastrsquares Solution. 


[U,S,V] = Ssvd(A3) ; 
U=U(: 1Ir)jVvV=V(1: 


AIP = vx*diag(1./diag 





(证 ) To see that the reslidual error of thlis solution ls less than that of 
any other vector around lt, type the following statements into the 
MAILAB command window. What is Implied py the result? 


err = norm(A3*Xx-b3) 
forn = 1:1000 
if norm(A3* (x+rand(Size(Xx))-0.5)-b)<err 


disp('What the hel1 smaller error SolL - not LSE? ) |; 
end 
end 





(dJ) For the problem 


下 昌 
5 9 和 2 
A4X 一 7 11 _1] X2 一 3 一 by4 (P2.8.0) 
三 2 | 4 


compare the LS solution based on SVD and that obtained by Eq. (2.1.10). 

(cf) Ihis problem illustrates that SVD can be used for fabricating a _ universal 
Solution of a set of linear equations, minimum-norm or least-squares，for 
all the possible rank deficiency of the coefficient matrix 4. 
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2.9 Gauss-9eidel Iterative Method with Relaxation TIechnique 


(a) Try the relaxation technique (introduced in Section 2.3.3) with Several 
values of the relaxation factor w = 0.2, 0.4, ..…. ,1.8 for the following 
problems. Find the best one among these values of the relaxation factor 
for each problem，together with the number of iterations required for 
satisfying the termination criterion ||Xk4l 一 |/xkll < 10-. 


OAx 一 医 | 图 到 目 一 Di (P2.9.1) 


(4x 一 医 | 国 区 目 =b， (P2.9.2) 


(iii) The nonlinear equations (E2.5.1) given in Example 2.9. 


(b 


S 必 


Which of the two matrices 41 and 4 has stronger diagonal dominancy 
in the above equations? For which equation does Gauss-Seldel iteration 
Converge faster, Eq. (P2.9.1) or Eq. (P2.9.2)? What would you conjecture 
about the relationship between the convergence Speed of Gauss-Seidel 
iteration for a set of linear equations and the diagonal dominancy of the 
coefficient matrix 4? 


(Cc 


AR 


Is the relaxation technique always helpful for Improving the convergence 
Speed of the Gauss-Seidel iterative method regardless of the value of 
the relaxation factor w? 


INTERPOLATION 
AND CURVE FITTING 





There are two topics to be dealt with in this chapter, namely, interpolationl and 
curve fitting. Interpolation is to connect discrete data points in a plausible way 
So that one can get reasonable estimates of data points between the given points. 
The interpolation curve goes through all data points. Curve fitting,， on the other 
hand, is to fnd a curve that could best indicate the trend of a given Set of data. 
The curve does not have to go through the data points. In Some cases, the data 
may have different accuracy/reliability/uncertainty and we need the weighted 
least-squares curve fitting to process Such data. 


3.1 INTERPOLATION BY LAGRANGE POLYNOMIAL 


For a given set of N 十 1 data points {(xo, yo)， Co yD， ,Cov, yN)}]，we want 
to fnd the coefficients of an NWth-degree polynomial function to match them: 


PN 一 a0 十 QI 十 0X2 十 .十 GNXY (3.1.1) 


The_ coefficients can be obtained by solving the following System of linear 
equations. 
G0 十 X0G1 十 xld2 十 .十 X0aN 一 y0 


a0 十 X101 十 Xia2 十 .十 XYax 一 当 本 4 


2 N 
0G0 十 XYNGQI 十 XYNa2 十 :…… 十 XNGN 一 yN 


1 If we estimate the values of the unknown function at the points that are inside/outside the range 
of collected data points，we call it the interpolation/extrapolation . 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang, Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 儿 Sons, Inc. 
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But， as the number of data points increases，So does the number of unknown 
variables and equations，consequently, it may be not So easy to Solve. That is 
why we look for alternatives to get the coefficients {fao, al, ...，GN|. 

One of the alternatives ls to make use of the Lagrange polynomials 


CC 一 X1)C 一 X2) (一 XN) (人 一 X0)C 一 X2) (一 XN) 
AN 00) = MX 一 一 一 十) 一 一 一 
(xco 一 X1)(Xo 一 22) (CCXo 一 XN) (xl 一 X0)GXI 一 22) (CC 一 XN) 
(人 一 X0)C 一 Xp(GX 一 XN-1) 
0 (CN 一 X0)(CXCN 一 X1) (CN 一 XN_1) 
N N N 
[Le 一 Xp) 大 一 和 XK 
/ 一 历 玫 和 N 六 h 二 Nm 三 一 -一 (3.1.3 
wG) 2， 人 岂 元 G.13) 


It can easily be Shown that the graph of this function matches every data point 
LN(Con) 一 ymr VY71 一 0,1...，N (3.1.4) 


Since the Lagrange coefficient polynomial 友 wm (xz) is 1 only for x = wm and Zero 
for all other data points x 一 XK (天 1). Note that the Nth-degree polynomial 
function matching the given N 十 1 points ls unique and So Eq. (3.1.1) having 
the _ coefficients obtained from Eq. (3.1.2) must be the Same as the Lagrange 
polynomial (3.1.3). 

Now，we have the MAILAB routine“lagranp()”which finds us the coef- 
ficients of Lagrange polynomial (3.1.3) together with each Lagrange coefficient 
polynomial 了 wm (x). In order to understand this routine, you Should know that 
MAILAB deals with polynomials as their coefficient vectors arranged in descend- 
ing order and the multiplication of two polynomials corresponds to the convolu- 
tion of the coefficient vectors as mentioned in Section 1.1.0. 


function [1,L] = Jagranp(x,y) 

%Input : X [xoO x1 ... XN],yY= [yoyl ...YyN] 

%Output: 1 = Lagrange polynomial coefficients of degree N 
L = Lagrange coefficient polynomial 

1Length(x)-1; s#%the degree of polLlynomial 


1:N + 1 


机 
K 


OF = 1:N+1 
ifk-=mP=conv(P,[1 -x(k)])/(x(m)-x(k)); end 
end 
L(m,:) = P; %5Lagrange coefficient polynomial 
工 +y(m)x*P; #sLagrange polynomial (3.1.3) 
end 


#%5do_lagranp .m 

X [-2 -112];y= [-6006];5%5given data points 

下 Lagranp(x,y) 5 find the Lagrange polLynomial 

xx = [-2: 0.02 : 2]; yy = polyval(1,xx); s#%interpolate for [-2,2] 
ClLf，plot(xx;,yy，b' ，Xx，y，*') %5plot the graph 
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(2, 6) 























Figure 3.1 The graph of athird-degree Lagrange polynomial. 


We make the MAILAB program “do_lagranp.m” to use the routine 
“1agranp()”for fnding the third-degree polynomial /3(x) which matches the 
four given points 

长 一 2 一 自 ) (三 0)， ( 0)， (2 0)} 


and to check if the graph of /3(x) really passes the four points. The results from 
running this program are depicted in Fig. 3.1， 


>>do_lagranp 
1= 1 0 -1 05%meamningAoc) = 1:.x3+0:x2 一 1:.x 十 0 


3.2 INTERPOLATION BY NEWTON POLYNOMIAL 


Although the Lagrange polynomial works pretty well for interpolation 1irrespec- 
tive of the interval widths between the data points along the x-axls, it requires 
restarting the whole computation with heavier burden as data points are appended. 
Differently from this, the Nth-degree Newton polynomial matching the N 十 1 


data points {(xo, yo), (xl, y1),...，(CxNw, yw)} can be recursively obtained as the 
Sum of the (CN -- J)th-degree Newton polynomial matching the N data points 
{(xzo, yo), (xl 7 ,CN_l yw_1)} and one additional term. 


1HN(Z) 一 G0 十 QI 一 X0) 十 02( 一 X0)G 一 X1) 十 …， 


一 AN_I(C) 十 CN 一 X0)GX 一 X) CC 一 XN-1) with 0(xz) 三 Go0 
(3.2.1) 
JIn order to derive a formula to find the successive coefficients {ao, al,...，,QCN} 


that make this equation accommodate the data points，we will determine ao and 
al SO that 
11(X) 一 70Cx) 十 GCC 一 X0) (3.2.2) 
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matches the first two data points (xo, yo) and (xl, y1). We need to Solve the two 
equations 


11(X0) 一 00 十 G1(x0 一 2X0) 一 0 


11(X1) 一 00 十 GCCI 一 2X0) 一 当 


to get 

y1 一 00 y1 一 y0 
0 三 y0， RE = 玉 00 (3.2.3) 
X1 一 X0 X1 一 X0 


Starting from this frst-degree Newton polynomial, we can proceed to the Second- 
degree Newton polynomial 


12(XY) 一 1) 十 0 一 X0)CG 一 X1) 一 00 十 QI 一 X0) 十 ax 一 X0)G 一 2X1) 

(3.2.4) 
which，with the same coefficients ao and al as (3.2.3)，Sstill matches the first 
two data points (xzo, yo) and (xl, y1)，Since the additional (third) term is zero 
at (xo, y0) and (xl, y1). This ls to Say that the additional polynomial term does 
not disturb the matching of previous existing data. Therefore，given the addi- 
tional matching condition for the third data point (xz, y2)，we only have to 
Solve 


12(X2) 一 00 十 G1(X2 一 X0) 十 02(x2 一 X0)(x2 一 2X1) 三 六 


for only one more coefficient a2> to get 





史 0 二 

本 六 aa 一 zi 2 清二 疝 

六 一 一 一 一 一 一 
(xc2 一 X0)(x2 一 X1) (xc2 一 X0)(x2 一 XI) 


y1 一 y0 
入 二 说 征询 志 的 三 一 一 二 
X1 一 X0 


(xa2 一 X0)(x2 一 X1) 
2 一 7 J 包 一 加 
2 一 :1 X1 一 X0 刀 记 一 Dj 


和 全 和 一 万 态 (3.2.5) 
X2 一 X0 X2 一 X0 





Generallizing these results (3.2.3) and (3.2.3) yields the formula to get the Nth 
coefficient wx of the Newton polynomial function (3.2.1) as 


Do 一 太一 Do 一 广 


XN 一 X0 


QN 三 DAy 方 (3.2.0) 


This is the divided difference，which can be obtained successively from the 
Second TIow of Table 3.1. 
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Table 3.1 Divided Difference Table 


function [n,DD] = newtonp(x;,y) 
%5Input : X [x0O xf1 ..，XN] 
Vy [y0O y1 ..，yN] 
Output: n Newton polynomial coefficients of degree N 
Length(x)-1; 
+ TON + 1); 


y ， 


) 三 
+ 1 
:N+2- ks%Divided Difference Table 

(DD(Im + 1;k- 1) - DDmn;k - 1))/(xm+k - 1)- xm)); 


DD(1,:); %5Eq.(3.2.6) 
n=a(N+1); %Begin with Eq.(3.2.7) 
for kK=N:-1:1 %5Eq.(3.2.7) 
n= [na(k)] - [0 nx*x(k)]; %5n(x)*(X- X(k - 1))+ak -1 
end 





Note that, as mentioned in Section 1.3, it js of better computational efficiency to 
write the Newton polynomial (3.2.1) in the nested multiplication form as 


HN(z) 一 (CC…(CawC 一 XN-1) 十 GN-1)G 一 YN-2) 十 …) 十 GCC 一 X0) 十 ao0 
(3.2.7) 

and that the multipllication of two polynomials corresponds to the convolution 
of the coefficient vectors as mentioned in Section 1.1.0. We make the MAILAB 
Ioutine“newtonp()”to compose the divided difference table like Table 3.1 and 
construct the Newton polynomilal for a set of data points. 

For example, Suppose we are to find a Newton polynomial matching the fol- 
lowing data points 


{( 一 2， 一 0)， (一 |， 0)， (1， 0)， (2， 0)， (4， 00)} 


From these data points，we construct the divided difference tabple as Table 3.2 
and then use this table together with Eq. (3.2.1) to get the Newton polynomial 
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Table 3.2 Divided differences 





as follows: 


100) 一 MX 十 Dj 一 xz 十 及 矶 ce 一 xzo)Gc 一 2) 
+D5hHax 一 xoc 一 xz) 一 z) 十 0 
三 一 0 十 0GX 一 (一 人 ) 一 20 一 (一 2))Gx 一 (一 D) 
十 1Gx 一 (2))G 一 (一 D)G 一 了 
= -6+6C+2) 一 2c+2)c+D+CT+T202 一 1 
一 xz 十 (-2 二 2)x72 十 (6 一 6 一 Dr 一 6+12 一 4 一 2 一 妇 一 x 











We might begin with not necessarily the first data point，but， say, the third one 
(1;0), and proceed as follows to end up with the Same result. 


1 一 六 十 万 户 A 一 妇 ) 十 人 有 户 C 一 和 2)G 一 区 3) 
二 有 Paz)G 一 5)G 一 22) 二 0 
一 0 十 6 一 JJ) 二 7 一 TD 一 2) 二 1x 一 JJ) 一 2)G 一 4) 





=60 一 D 上 +702 一 3x+2) 二 0 一 3xr 十 2)(C 一 有 
=23+O0-T)xz2+(66-21+140xz 一 6+14 一 8 一 妇 一 : 


This process ls cast into the MAILAB program“do_newtonp .m7”"，which illus- 
trates that the Newton polynomial (3.2.1) does not depend on the order of the 
data points; that 1$，changing the order of the data points does not make any 
difference. 
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%5do_newtonp .m 


= [-2-1 1 6 601]; 


0 
newtonp (xy xy) for comparison 
0 


[-1 -2 1 6 60] ; 
n1 newtonp (X， 1 rder of data changed for comparison 
XX [-2:0.02: al(n,Xxx) ; 
CLf，plot(xx;,yy， 





Now, let us see the interpolation problem from the viewpoint of approximation. 
For this purpose, Suppose we are to approximate Some function，Say， 


1 


二 


by a polynomial， We first pick up some Sample points on the graph of this 
function, Such as listed below, and look for the polynomial functions 4(Xx), 18(x)， 
and 110(x) to match each of the three sets of points, respectively. 


国王 加 杞 本 加 IDD 


反 | -10| -075 | -05| -025|0| 025 | 05 | 075 | 10| 


四 EDOEIEENEGOGIII3IEIIID 


We made the MAILAB program“do_newtonp1.m” to do this job and plot the 
graphs of the polynomial functions together with the graph of the true function 
J(x) and their error functions Separately for comparison as depicted in Fig. 3.2， 
Where the parts for mg(x) and 17110(0x) are omitted to provide the readers with 
Some Toom for practice， 











s#%do_newtonp1.m 一 plot Fig.3.2 

Xx= [-1 -0.5 0 0.5 1.0];y= f31(x); 

n newtonp(x,y) 

xx = [-1:0.02: 1]; %the interval to Look over 

yy = f31(xx); #%sgraph of the true function 

yy1 = polyval(n,Xxx); %graph of the approximate polynomial function 


Subplot(221)，plot(xx,yy，kKk-` ，Xiy，0'，Xxyy1，b ') 
Subplot(222)，plot(xx,yy1-yy， mr ) %5graph of the error function 


function y = f31(Xx) 
y=1./(1+8*Xx.^2); 
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0.5 上 mio(X) - XX) ] 





























(al) 4/8/10th -degree polynomial (b) The error between the approximating 
approximation polynomial and the true function 


Figure 3.2 Interpolation from the viewpoint of approximation. 


了 R 了 emark 3.1. Polynomial Wiggle and Runge Phenomenon. Here is one thing to 
note. Strangely, increasing the degree of polynomial contributes little to reducing 
the approximation error. Rather contrary to ouUT usual expectation, it tends to make 
the oscillation strikingly large, which is called the polynomial wiggle and the error 
gets bigger in the parts close to both ends as can be Seen in Fig. 3.2，which is 
called the Runge phenomenon. That is why polynomials of degree 3 or above are 
Seldom used for the purpose of interpolation, unless they are Sure to fit the data. 


3.3 APPROXIMATION BY CHEBYSHEV POLYNOMIAL 


At the end of the previous Section，we considered a polynomial approximation 
problem of fnding a polynomial close to a given (true) function 太 x) and have 
the freedom to pick up the target points {xo, xl ...,XNj in our own way. Once 
the target points have been fxed, it ls nothing but an interpolation problem that 
can be Solved by the Lagrange or Newton polynomial. 

In this section，we will think about how to choose the target points for better 
approximation, rather than taking equidistant points along the x axis. Noting that 
the error tends to get bigger in the parts close to both ends of the interval when 
we chose the equidistant target points, lt may be helpful to set the target points 
denser in the parts close to both ends than in the middle part. In this context,， a 
possible choice is the projection (onto the x axis) of the equidistant points on the 
circle centered at the middle point of the interval along the x axis (See Fig. 3.3)， 
That 1s，we can choose in the normallized interval [一 1, 十 H 


2V 二 1 一 和 KK 
RS for 天 一 0,1...，N (3.3.1a) 
2(V 十]) 


and for an arbitrary interval [C, D]， 


b 一 a ， a 二 bb  b 一 a 2N 十 1 一 2K a 十 b 
Xk 一 一 Xk 十 一 一 三 一 一 50S 一 一 一 一 世 十 
2 人 2(N 十 ]) 人 





fork 一 0,1,...,N 
(3.3.1b) 
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-| 人 


X0= X1 = 0 X3 三 X4 = 
cos 9r/10 cos 7/10 X2= cos 57r/10 cos 3rr/10 cos mr/10 


Figure 3.3 ”Chebyshev nodes (N = 4). 


which are referred to as the Chebyshev nodes. The approximating polynomial 
obtained on thebasis ofthese Chebyshev nodes is calledthe Chebyshev polynomial. 
Let us try the Chebyshev nodes on approximating the function 


1 


0 


We can Set the 3/9/11 Chebyshev nodes by Eq. (3.3.1) and get the Lagrange 
Or Newton polynomials c4(x), cg(x)，and clio(x) matching these target points， 
which are called the Chebyshev polynomial. We make the MAILAB program 
“do_lagnewch.m” to do this job and plot the graphs of the polynomial functions 
together with the graph of the true function 太 (x) and their error functions Sep- 
arately for comparison as depicted in Fig. 3.4. The parts for cg(x) and clo(x) 
are omitted to give the readers a chance to Practice what they have learned in 
this Section. 


%5do_lagnewch.m 一 plot Fig.3.4 

N=4;jk= [0O:N]; 

X=coSs((2*N+ 1 - 2*k)*pi/2/(N + 1)); #%Cchebyshev nodes(Eq.(3.3.1)) 
y=f31(Xx) ; 

C=newtonp(x,y) s#%sChebyshev polLynomial 


xx = [-1:0.02: 1]; %the interval to 1ook over 

yy f31(Xx); #%sgraph of the true function 

yy1 = polyval(c,xx); %graph of the approximate polynomial function 
Subplot(221)，plot(xx;,yy，k-` ，Xxiy，0'，Xxiyy1，b') 
Subplot(222)，plot(xx,yy1-yy， mr ) %graph of the error function 





Comparing Fig. 3.4 with Fig. 3.2, we see that the maximum deviation of the 
Chebyshev polynomial from the true function is considerably less than that of 
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(a) 4/8/10m -degree polynomial approximation (b) The error between the Chebyshev 
approximating polynomial and the true function 


Figure 3.4 Approximation using the Chebyshev polynomial. 


Lagrange/Newton polynomial with equidistant nodes. It can also be Seen that 
increasing the number of the Chebyshev nodes 一 or，equivalently，increasing 
the degree of Chebyshev polynomial 一 makes a Substantial contribution towards 
reducing the approximation error, 

There are several things to note about the Chebyshev polynomial. 


了 R 了 emark 3.2. Chebyshev Nodes and Chebyshev Coefficient Polynomials 7 (xD) 
1. The Chebyshev coeffticient polynomial is defined as 
TwiiC ) 一 cos((NW 二 lcosrlxr)  ， for-1<x <+l (3.3.2) 


and the Chebyshev nodes defined by Eq. (3.3.1a) are actually zeros of this 
function: 


TwiiGC ) = cos((V 十 1)cos-Lr)=0，(V+lDcos-x = (2K 十 Dr/2 


2. Equation (3.3.2) can be written via the trigonometric formula in a Tecursive 
form as 


TNwiCc ) 一 cos(cos-1 2X 十 Nocosr-l 2 ) 
一 cos(cos-1 2 ) cos(N cos-1 2 ) 一 Sin(cos-1 7) sin(N cos-1 2 ) 


王 X TvOc 十 zlcos(N 十 J)cos-: 2 ) 一 cos((V 一 Tcosr-: 7)} 


1 1 
一 XY7TNCx ) 十 本 INw+1IG ) 一 本 7w-IG ) 


Two )=2xz7wo) 一 Tv ix) for NW>1 (3.3.3a) 


TO) 一 cos0 = 1， Tc ) 一 cos(cos-1! 7) 二 (3.3.3b) 


3. At the Chebyshev nodes 2 defined by Eq. (3.3.1a), the set of Chebyshev 
coefficient polynomilals 


{meD mo) Two)} 
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are orthogonal in the sense that 


NV 
六 CT =0 formmn 关 六 


(3.3.4al) 
大 一 0 
N 
汪 T2(xu) 一 一 for 刀 尖 0 (3.3.4b) 
大 一 0 
N 
>》 10=N+l form =0 (3.3.4c) 
大 一 0 


4. The Chebyshev coefficient polynomials Tw+iCr ) forup to N = 6 are col- 
lected in Table 3.3,，and their graphs are depicted in Fig. 3.5. As can be 
Seen from the table or the graph, the Chebyshev coefficient polynomials of 
even/odd degree (NW 十 ]) are even/odd functions and have an equi-ripple 
characteristic with the range of [一 1 十 ]Hj, and the number of rising/falling 
(intervals) within the domain of [一 1 十 1H] is N 十 1. 


We can make use of the orthogonality [Eq. (3.3.4)] of Chebyshev coefficient 
polynomials to derive the Chebyshev polynomial approximation formula. 


N 
/会 痪 国 二 和 人 到 
弃 1 “二 4 (- 2 ) 


(3.3.3) 


















































2 T 1 T 
1 T 0 ] 
上 
号 0 1 
(al To (oOD)=1 
1 
0 让 车 
-1 1 
-1 0 1 
(c) 72 CO) 
1 1 T 
0 0 
三 1 -1 1 
-1 0 1 -1 0 1 


(e) 74 (X0) () 75 (x) 


Figure 3.5 ”Chebyshev polynomial functions. 
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Table 3.3 Chebyshev Coefficient Polynomials 


To )=1 

站)=x Oo avariable normalized onto [一 1, 1]) 
DC )=2x2 一 1 

DC ) 一 4x3 一 3x 

TCD 一 8x4 一 8x2 十 1 
TO) 一 16x5 一 20x3 十 SX 

75( ) 一 32x56 一 48x4 十 18x 2 一 1 

DOx) 一 64x 7 一 112x5 十 56x3 一 TX 











where 


四 = 六 nd ie 间 (3.3.6a) 


dm=TTT 过 CDTn Ce 


11(2N 十 1 一 20) 
一 -一 一 一 一 一 ff 三 2 
-reomn XNTT 元 OT 111 


(3.3.6b) 


function [c,x;,y] = cheby(f,N,ab) 
%Input : 于 function name on [ay,bj] 
%0Output: C Newton polynomial coefficients of degree N 
5 (X,y) = Chebyshev nodes 
if nargin == - 2 ， a= -ljb=1; end 
= [0: N]; 
theta = (2*N+ 1 - 2*k)x*xpi/(2*xN + 2); 
xn = Cos(theta) ; 
(b - a)/2*xn +(a + b)/2; 

y = feval(f,Xx); 
d(1) = yx*oneSs(N + 1,1)/(N+1) 
form=2:N+1 

coSs_mth = cos((m-1)*theta) ; 

d(m) = yx*cos_mth' *2/(N + 1); %Eq.(3.3.6b) 


%the inverse of (3.3.1b) 
%Eq.(3.3.3b) 
%Eq.(3.3.5) 


2x*conv(xn,T 1) -[0 0 T0]; sEq.(3.3.3a) 
tmp ; 
cl] + d(m)xT_ 1; %Eq.(3.3.5) 
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We can apply this formula to get the polynomial approximation directly for 
a given function jx)，without having to resort to the Lagrange or Newton 
polynomial. Given a function, the degree of the approximate polynomial, and the 
leftyright boundary points of the interval, the above MAILAB routine“cheby ()” 
uses this formula to make the Chebyshev polynomial approximation. 

The following example illustrates that this formula gives the Same approximate 
polynomial function as could be obtained by applying the Newton polynomilal 
with the Chebyshev nodes， 


了 上 xample 3.1. Approximation by Chebyshev Polynomial. Consider the Problem 
of fnding the Second-degree (N = 2) polynomial to approximate the function 
FG) =1/G +8x2). We make the following program “do_cheby.m?, which uses 
the MAILAB routine“cheby()”for this job and uses Lagrange/Newton polyno- 
Imial with the Chebyshev nodes to do the Same job. Readers can run this program 
to check if the results are the Same， 


s%5do_cheby .m 

N=2ja=-2;jb= 2 

[c,xl,y1] = cheby('f31' ,Nab) %5Chebyshev polLlynomial ftn 
sfor comparison with Lagrange/Newton polLlynomial ftn 


K [0:N]; xn = cos((2*sN+ 1 - 2*k)*pi/2/(N + 1));#Eq.(3.3.1a):Chebyshev nodes 
X ((b-a)*xn +a + b)/2; ss%Eq.(3.3.1b) 
y f31(x); n = newtonp(x,y)，1 = lagranp(x,y) 





>>do_cheby 
c = -0.3200 -0.0000 1.0000 


3.4 PADE APPROXIMATION BY RATIONAL FUNCTION 


Pade approximation tries to approximate a function 太 (x) around a point zx” by a 
rational function 


本 


一 二  - wihM=NorM=N+l 
DNGz 一 X2) 


PMNGC 一 并 ) 
人 
1 十 四 G 一 X9) 十 几 人 一 X 帮 十 十 dvGE 一 X0)A 
G.4.0 
where F(x2),， 广 (2)，FO(x2) .FUT+N(O2) are known. 


How do we find such a rational function27 We write the Taylor series expansion 
of 太 (x) up to degree M 十 Natxy 一 2 as 
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Jo sTirwG 一 xz2) = Fe) 十 广 Cco)GCc 一 x0) 
0) (x2) UL+N)(C2) 
了 CQ 二 NI 


一 aa 一 xx 十 ooG 一 xz 十 :十 NG 一 xz)2TY (3.4.2) 


(cc 一 X2)2 十 .十 ( 二 MT 


Assuming x? = 0 for Simplicity，we get the coefficients of DN(xz) and ONOxz) 
Such that 


Orx) 
一 0 
DN(x) 





TMr+N(CC) 一 


(Co 十 QIX 十 :十 QMHNX AT )GL 二 dX 十 十 dvx) 
一 (Co 二 dz 十 十 guxx 7”) 











三 : 
1 十 QIX 十 0 十 :十 QNXX 
(Co 二 OILX 十 :十 GUNHNX TI 二 GY 十 十 dvx”) 
一 go 十 gIX 十 :二 OA (G3.4.3) 
by solving the following equations: 
00 二 940 
QI1 十 aodi 二 41 
02 十 Q1G1 十 aod2 一 qg2 (3.4.4a) 
QM 十 NM-1dl 十 CQM-202 十 ap-Ndvw 三 9gM 
aM+1L 十 ad 十 GM-102 … ”十 Ga-N+ldw = 三 0 
De 
CQM+N 十 GNMH+N_IdI 十 QH+N_2d2 十 GdN 一 0 
Here, we must first Solve Eq. (3.4.4b) for di ,... ,dx and then substitute ds 


into Eq. (3.4.4a) to obtain go, gl1,..…. ,9gM， 

The MAILAB routine“padeap ()”implements this scheme to find the coef- 
fcient vectors of the numeratordenominator polynomial OOxJ)/ADNC) of the 
Pade approximation for a given function jx). Note the following things: 


。The derivatives _ 广 (x2)，FO(x2)，...，FUW+N(2) up to order (M 十 N) are 
computed numerically by using the routine“difapx()”, that will be intro- 
duced in Section 5.3，. 

。 JIn order to compnute the values of the Pade approximate function， we Substi- 
tute (X 一 X?) forx in PN(C) which has been obtained with the assSumption 
that x?” 一 0. 
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function [num,den] = padeap(f,xo,M,N,X0,Xxf) 
%Input :下 = function to be approximated around in [xo，xf] 
soOutput: num = numerator coeffs of Pade approximation of degree M 
55 den = denominator coeffs of Pade approximation of degree N 
a(1) = feval(f,xo); 
= .01; tmp = 1; 
for II= 1:M+N 
tmp tmp*iIxh; 和 LI1h“^ 
dix = difapx(I,[-II])*feval(f,xo+[-I:I]*h) ; %dqerivative(Section 5.3) 
al(i 1) = dix/tmpj %Taylor series coefficient 
end 
form= 1:N 
n= 1:Ni Amn)= alM+1+m- mn) 
b(m) = -a(M+ 1 +m); 


d=Ab'; %5Eq.(3.4.4b) 
form=1:M+1 
mm = minm - 1;N); 
dq(m) = am:-1:m - mm)x[1; d(1:mmn)]; #%Eq.(3.4.4a) 
end 
num=qM+1:-1:1)/d(N); den= [d(N:-1:1)” 1]/d(N); %descending order 
If nargout == s% plot the true ftn，Pade ftn and Taylor expansion 
if nargin < 6,，Xx = xo - 1; xf= xo+ 1; end 
Xx = XO+[xf-x0]/100*[0:100]; yt = feval(f,x); 
xl1 = X-xoj yp = polyval(num,x1)./polyval(den,x1) ; 
yT = polyval(a(M+N+ 1:-1:1),x1); 
Clf， plot(x,yt，Kk ， Xiyp， rr ， Xi;yT，b ) 





Example 3.2. Pade Approximation for Fx) = e“. Lets find the Pade approx- 
imation P32(X) 一 0O3()/D2(x) for jx) 三 ex around x?” 一 0. We make the 
MAILAB program“do_pade.m"，which uses the routine“padeap()”for this 
job and uses lt again with no _ output argument to See the graphic results as 
depicted in Fig. 3.0. 


>>do_pade s%Pade approximation 
n=0.3333 2.9996 11.9994 “19.9988 
d = 1.0000 -7.9997 “19.9988 


s%5do_pade.m to get the Pade approximation for f(x) 
f1 = inline(' exp(x) ，X'); 

3;N=2; %the degrees of Numerator Q(x) and Denominator D(Xx) 
Xxo = 0; %the center of Taylor Series expansion 


[n,d] = padeap(f1,xo,M,N) s%to get the coefficients of Q(x)/P(Xx) 
x0 = -3.5; xf = 0.5; %left/right boundary of the interval 
padeap (f1,xo,M,N,X0,Xxf) %to See the graphic results 





To confirm and support this result from the analytical point of view and to help 
the readers understand the internal mechanism，we perform the hand-calculation 
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Figure 3.6 ”Pade approximation and Taylor series expansion for fg = ex(Example 3.2.). 


procedure. First，we write the Taylor series expansion at Y 一 0 up to degree 
MT+TN=5Sforthe given function 太 x) 一 ex as 


MT 二 N 朋 
二 大 (CO) 大 1 2 1 汪 1 4 1 5 
7y(Cx) 一 汪 5 一 十 x 十 5 十 可 十 下 二 SIX 十 …. 
(上 E3.2.1) 
whose coefficlents are 
一 1 一 1 一 兰 三 三 (上 3.2.2) 
CC0 一 上 ， 01 一 ]， 4 一 3， 4 一 全 44 二 54， 4 一 120… 和 


We put this into Eq. (3.4.4b) with M =3,N=2 and solve it for ds to get 
DO) 一 1+dx 十 dx 


+ad+oa=0 LU6 12]1Tw] 『 -124 d] _『「 -2/5 
中 +od+az=0 1124 16 一 | -1U120| | 到 | 二 | cl/20 
(E3.2.3) 


Substituting this to Eq. (3.4.4a) yields 


40 三 ao 三 1 
dl1 一 0 十 aodl 王 1 十 1x(-2/3) = 3/5 
4 一 二 ad 十 ao =1/2 二 1x(-2/5) 十 1x(1/20) = 3/20 


0 二 0+od+a 吕 =1/6+(112) x(-2/3) 二 1x (1/20) = 1/60 
(E3.2.4) 
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With these coefficients，we write the Pade approximate function as 


23C) 1+(G/3)xz 十 G/20)x7 十 (1/60)x” 


Da3.2(X) 一 DO 号 1 十 (一 2/S)xz 十 (1/20)X2? 
(L/3)xz 十 3x“ 十 12x 十 20 
_ (CU3) 妇 十 3 说 十 12x 十 20 E3.2. 
X2 一 8x 十 20 We 


3.5 INTERPOLATION BY CUBIC SPLINE 


开 we use the Lagrange/Newton polynomial to interpolate a given Set of NW 十 1] 
data points, the polynomial is usually of degree N and so has N -- 1 local extrema 
(maxima/minima). Thus, it will show a wild Swing/oscillation (called "polynomial 
WwWiggle"), particularly near the ends of the whole interval as the number of data 
points increases and So the degree of the polynomial gets higher, as lllustrated 
in Fig. 3.2. Then，how about a piecewise-linear approach，like assigning the 
individual approximate polynomial to every Subinterval between data points? 
How about just a linear interpolation 一 that 1S$，connecting the data points by 
a Straight line> It is So Simple，but too Short of Smoothness. Even with the 
Second-degree polynomial, the piecewise-quadratic curve ls not Smooth enough 
to please OUT eyeSs,，Since the Second-order derivatives of quadratic polynomials 
for adjacent Subintervals can't be made to conform with each other. In real 
life, there are many cases where the continujity of Second-order derivatives js 
desirable. For example, it is very important to ensure the Smoothness up to order 2 
for interpolation needed in CAD (compnuter-aided design)/CAM (computer-aided 
manufacturing), computer graphic, and robot path/trajectory planning. That's why 
We often resort to the piecewise-cubic curve constructed by the individual third- 
degree polynomials assligned to each Subinterval, which is called the cubic spline 
interpolation. (A Spline is a kind of template that architects use to draw a smooth 
Curve between two points.) 

For a given Set of data points {(xck yi) 大 一 0:NWj， the cubic Spline y(xz) 
conslsts of N cubic polynomial sk(xz) SS assligned to each Subinterval Satisfying 
the following constraints (SO)-(S4). 


(SO) so = CD = SC 一双 靖 十 Sa 一 区 十 SG 一 x) 十 So0 
for x E [Xe Xe 一 0:N 

(S1]) SK (OZ 一 9K.0 一 )K for 一 0:N 

(S2) SI =SOE) 一 0 一 次 forK=1:N 一 1 

(S3) S4_1CXH) 三 SC) 一 OK 1 forK 上 = 王 1:N 一 1 

(S4) SITG) 三 SOC) 一 2Sk2 for 大 一 1:N 一 1 


These constraints (S1)-(S4) amount to asetof VW 十 1+3MN 一 IJ)=4N 一 2 
linear equations having 4N coefficients of the N cubic polynomials 


{9ko ok ok ok3 一 0:N 一 二 
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Table 3.4 _ Boundary Conditions for a Cubic Spline 


人 First-order derivatives 。 s0(Cxo) 一 901, SNCN) 一 SN 
specified 

(ii) Second-order 3s0 (xz0) 一 2802,sSN(XN) 一 29N2 
derivatives Specified 
(end-curvature adjusted) 


万 
(iii) Second-order derivatives 81 (xo) 三 5 (xz1) 十 一 (CsYCom) 一 sy(x2)) 
extrapolated 名 玉 
SENEE NTOCNID) 十 





-1 A/ A/ 
万 (CN_1ICCN-1) 一 SN_2(CXCN-2)) 
N-2 


as their unknowns. Two additional equations necesSsary for the equations to be 
Solvable are Supposed to come from the boundary conditions for the firstsecond- 
order derivatives at the end points (xo, yo) and (xNw, yw) as listed in Table 3.4. 

Now,noting from (S1) that So 王 次 , we will arrange the constraints (S2)-(S4) 
and eliminate 9kl, ok37S to Set up a set of equations with respect to the NW 十 1] 
unknowns {S2, 大 一 0:N)}.Iorder to do So, we denote each interval width by 
太一 Xi+l 一 2 and Substitute (SO) into (S4) to write 


SO) 一 6S3 大 十 2S52 三 SIC 一 29k+12 


1 
5.37K 一 了 (St+12 ok2) (3.3.1a) 
1 
Sk-1.37-1 一 了 (ok2 一 912) (3.5.1b) 
We Substitute these equations into (82) with 上 十 1 in place of 天 


SC 一 SG 一 2 十 Sa 一 大 二 SIC 一 x) 十 So 三 ytH 


3 2 本 
SkK3 大 十 927 十 SI 十 次 三 HH 
to eliminate 3 S and Tewrite it as 


yk+1l 一 其 


万 
二 (Se 一 2) 十 9k21 十 ol 一 一 Gy 
3 太太 
jxk(CokH1L2 十 29k2) 十 39k1 一 3d (3.5.2a) 
ji_1(9k2 十 29 12) 十 3 1 三 30Cy 1 (3.5.2b) 


We also Substitute Eq. (3.5.1b) into (S3) 


SLC) 一 3S5 131 十 2S oj 1 十 SI EC 一 1 
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to write 


Sk1 一 9 三] 1 2 一 9 12) 十 211 1 12 一] 1 十 St 12) 
(3.3.3) 
In order to use this for eliminating 1 from Eq. (3.3.2)，we Subtract (3.5.2b) 
from (3.5.2a) to write 


ji(SkH12 十 2952) 一 1 12 十 295 12) 十 3(001 一 9 三 3(O 一 Gy 1) 
and then Substitute Eq. (3.3.3) into this to write 


HKECSEHL2 十 29k2) 一 -1092 十 2 -12) 十 31109k2 十 9 -12) 
三 3 一 GD) 
jj-195 12 十 2 1 十 Pi)Sk2 十 Pt 三 3 一 Gy 1D) (3.5.4) 
forK 王 1:WNW 一 1 


9ince these are N 一 1 equations with respect to NM 十 1 unknowns {Sk2, 大 三 0: 
N}， we need two more equations from the boundary conditions to be given as 
listed in Table 3.4. 

How do we convert the boundary condition into equations? In the case where 
the first-order derivatives on the two boundary points are given as () in Table 3.4， 
we Write Eq. (3.3.2a) for 上 一 0 as 


10(S12 十 202) 十 3S01 一 3dy0， 210402 十 1o912 一 3(dy0 一 90.1) 
(3.5$.9a) 
We also write Eq. (3.$.2pb) for 上 一 N as 


jiN -1(9N2 十 29Nw 12) 十 39N 11 一 3dyN-l 
and Substitute (3.5.3)(K = W) into this to write 


1N_1(SN2 十 29N 12) 十 39N1 一 31N 10N2 十 9N 12) 一 3dyN-l 
PN_1SN_12 十 27N_1SN2 一 3(0SN1I 一 Gyw_l) (3.5.Sb) 
Equations (3.3.9a) and (3.3.3b) are two additional equations that we need to solve 
Eq. (3.3.4) and that's it. In the case where the second-order derivatives on the 


two boundary points are given as (i) in Table 3.4， So and SN2 are directly 
known from the boundary conditions as 


So2 一 s0 (xzo)/2， SNw2 一 SN(CCN)/2 (3.5.0) 
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and, Subsequently, we have just N 一 1 unknowns. In the case where the Second- 
order derivatives on the two boundary points are given as (iii) in Table 3.4 


万 
5 Go) 三 sCo) 十 页 Gfe) 一 史 Co)) 





用 N_1 / 1/ 
瑟 (SN_1(CCN_1) 二 SNV_2(XN_2)) 
N 一 2 


SNCN) =SN_ICCN-1) 十 

we can instantly convert these into two edquations with respect to 9%o.2 and 9N.? 3aS 
1100.2 一 (po 十]1)912 十 170922 一 0 (3.5.7a) 

有 PN-29N2 一 (PN-1 十 PN-2)SN_12 十 PN_-19N 22 一 0 (3.5.7b) 


Finally，we combine the two equations (3.3.9a) and (3.3.3b) with Eq. (3.3.4) 
to wiite it in the matrix-vector form as 





270 10 0 @ @ 0.2 

10 2(po 十 ji) 7 。 。 91.2 | 
0 a 0 2 
。 。 ]N-2 2(N-2 十 PN-1) PN-1l Sa 
。 。 0 刀 N-1 27N-1 SN,2 


3(dyo 一 90.1) 
3(dyl 一 dy0) 
一 。 (3.3.8) 
3(QyN-1 一 ov | 


3(9N1 一 GyN-1) 


After solving this System of equation for {Sk2 ,大 一 0: N)，we Substitute them 
into (S1), (3.5$.2), and (3.3.1) to get the other coefficients of the cubic spline as 


(3.5.2) 


(S1 大 (G3.5.0 9k+12 一 92 
SOk.0 写 ， ETI 三 dx 一 本 (erL2 十 29k2)， 63 一 一 


3.3.9 
5 册 (3.5.9) 


The MAILAB routine“cspline()”constructs Eq.(3.3.8), solves it to get the 
cubic Spline _ coefficients for given x, y coordinates of the data points and the 
boundary conditions，uses the mkpp() routine to get the piecewilse polynomial 
expression, and then uses the ppval() routine to obtain the value(S) of the piece- 
wise polynomial function for xi 一 that is, the interpolation over xi. The type of 
the boundary condition is Supposed to be Specified by the third input argument 
KC. In the case where the boundary condition ls given as (1 im Table 3.4， 
the input argument KC Should be set to L/2 and the fourth and fifth input argu- 
ments must be the firstsecond derivatives at the end points. In the case where 
the boundary condition is given as extrapolated like (iii) in Iable 3.4, the input 
argument KC Should be set to 3 and the fourth and fifth input arguments do not 
need to be fed. 


INTERPOLATION BY CUBIC SPLINE 137 


function [yi,S] = cspline(x,y, Xi，KC,dqy0,dyN) 
#%This function finds the cubic splines for the input data points (xy) 
s%Input: X= [xoO xl ... xNl,，y= [yo0Oyl ...VyN]，xi=interpolation points 
KC = 1/2 for 1st/2nd derivatives on boundary Specified 
KC =3Tfor 2nd derivative on boundary extrapolated 
dy0 = S'(x0) = S01: initial derivative 
dyN = S'(xXN) = SN1: final derivative 
#%Output: Sn;k);jn= 1:N，k= 1,4 in descending order 
if nargin < 6， dyN = 0; end，if nargin < 5，dy0 = 0;j end 
if nargin < 4， KC = 0; end 
= length(x) - 1; 
55 ConStructs a Set of equations w.rt,， {S(n,2)，n= 1:N+ 1)} 
= ZeroS(N+ 1;N+ 1);)b= zeros(N + 1,1); 
ZeroS(N + 1,4); s% Cubic Spline Coefficient matrIX 
= 1:Ni hk) =Xxk+ 1) - x(k)j dy(k) = (y(k+ 1) -yGk))VnOk) 
Boundary condition 
if KC <= 1 5%lst derivatives Specified 
A(1,1:2) = [2*h(1) h(1)];b(1) = 3*(dy(1) - dy0); ssEdq.(3.5.5a) 
A(N+ 1ININ+1) = [h(N) 2*h(N)]; blN+ 1) = 3*(dyN - dy(N));sEq.(3.5.5b) 
elSseif KC == #%2nd derivatives Specified 
A(1;,1) = 2; b(1) = dy0; AIN + 1;N+1) = 2; b(N + 1) = dyN; %Eq.(3.5.6) 
else %52nd derivatives extrapolated 
A(1,1:3) = [h(2) - h(1) - h(2) h(1)] 
AN 1,N-1:N+1) = [h(N) - h(N)-hN - 
end 
form= 2:N %Eq.(3.5.8) 
Almm- 1:m+1)= [hm- 1) 2*(hm- 1) +hom)) hom)]; 
b(m) = 3*(dy(m) - dy(m - 1)); 
end 
S(:, 3) = AN\bi 
%5 Cubic spline coefficients 
form= 1:N 
S(m,4) = (S(m+1;,3)-S(m,3))/3/h(m); s%5Eq.(3.5.9) 
S(m,2) = dy(m) -h(m)/3*(S(m + 1，3)+2xS(m;3)); 
Sm,1) = y(m) 3 
end 
S=S(1:N，4:-1:1);) %dqescending order 
pp = mkpp(x,S); %make piecewise polynomial 
Vy ppval(pp,Xxi); %values of piecewise polynomial ftn 





(cf) See Problem 1.11 for the usages of the MAILAB routines mkpp() and ppval(). 


上 Example 3.3， Cubic Spline. Consider the problem of fnding the cubic spline 
interpolation for the NW 十 1=4 data points 


{(0, 0) (1 1D， (2 4)， (3 5)} (3.3.1) 
Subject to the boundary condition 
S0(X0) 一 30(0) 一 001 一 少 ; SN(XKN) 一 13(3) 一 13.1 三 : (上 E3.3.2) 


With the subinterval widths on the x-axls and the first divided differences as 
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= 一 一 1，d= 一 3， 一 二 一 1(E333) 


we Wiite Eq. (3.3.8) as 


2 1 0 | | 3(dyo 一 90.1) _3 
由 53.2 3(931 一 Gy1) 3 
Then we solve this equation to get 
05 三 二 3 二 3 二 二 55 三 3 (E3.3.5) 
and substitute this into Eq. (3.9.9) to obtain 
0.0 三 0， 91.0 一 1， 20 一 4 (E3.3.0) 


万 ] 

0,1 一 dy0 加 十 240.2) 一 ] 一 3G 十 9 (一 3)) 三 过 (上 E3.3.7a) 
1 ] 

1.1 一 dy1 2 本人 2 十 2491.2) 一 3 一 人 十 2 X 3) 三 忆 (上 E3.3.7b) 


相 1 
9 1 一 dy 一 本 (CS +292) 一 1 一 5G 十 2x (-3) = 2 (E3.3.79) 
5L2 一 2 3 一 (一 3) 





Sa 一 一 一 一 了 了 上 3.3.8 
370 3 
9022 一 2 一 3 一 3 
人 二 二 一 一 一 E3.3.8b 
3 371 3 
65 一 2 3 一 于 3) 
人 E 上 3.3.8 
人 3712 3 4 人 


%do_csplLlines .nm 

KC=1;dy0=2;j dyN=2; swWith Specified 1st derivatives on boundary 
x=[012 3; y=[0 1 4 5]; 

xi= Xx(1)+[0:200]*(x(end)-x(1))/1200; s%intermediate points 

[yi,S] = cspline(x,y, xi,KCc,dy0,dyN); S %cubic spline interpolation 
ClLf，plot(x,yY，ko' ,Xiyyi，k: ) 


yiI = Spline(x,[dyo y dyN],xi); #%sfor comparison with MATLAB built-in ftn 
hold on，pause，plot(x,y，ro' ,xi yi r: ) 

yiI = Spline(x,y,Xi); #%for comparison with MATLAB built-in ftn 
pause，plot(x,y，bo' ,xiyi，b') 

KC = 3;j [yi,S] = cspline(Xx,y,Xi,KC);%with the 2nd derivatives extrapolated 
pause，plot(x,yY，ko' ,xi yi，k') 
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0 0.5 1 1.5 2 2.5 3 
Figure 3.7 ”Cubic splines for Example 3.3. 


Finally，we can write the cubic Spline equations collectively from (SO) as 


SoC) 一 903G 一 zx0) 十 So2Cc 一 xz0) 十 SolG 一 xzo) 十 Soo0 
一 2x3 一 3x2 十 2x 十 0 

SC) 一 3 一 xD 十 SoC 一 2 十 SG 一 X) 十 Si0 
一 -2 一 1D3+3GC 一 1D2+2C 一 TD+1 

5200 一 930 一 2 十 go 一 2 十 SG 一 2) 十 90 
=20 一 2 一 3 一 2) 二 20 一 ID 十 4 


We make and run the program“do_csplLines.m7"，which uses the routine 
“cspline()”to compute the cubic spline coefficients {Sk3，9k2，9k1， ok0, 大 一 
0:N--]1andobtain the value(S) of the cublic spline function for xi (le., the 
interpolation over xi) and then plots the result as depicted in Fig. 3.7. We also 
compare this result with that obtained by using the MAILAB built-rin function 
“spline(x,y,xi)”, which works with the boundary condition of type () for the 
Second input argument given as [dy0 y dyN], and with the boundary condition 
of type (lin for the same lengths of x and y. 


>>do_csplines s%cubic splLine 
S = 2.0000 -3.0000 2.0000 0 
-2.0000 3.0000 2.0000 1.0000 
2.0000 -3.0000 2.0000 4.0000 


3.6 HERMITE INTERPOLATING POLYNOMIAL 


In Some cases，we need to find the polynomial function that not only passes 
through the given points，but also has the Specified derivatives at every data 
point， We call such a polynomial the Hermite interpolating polynomial or the 
oSculating polynomial. 
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For Simplicity，we consider a third-order polynomial 
jz) = 砚 妇 十 素 刀 十 页 * 十 硬 (3.6.1) 


matching just two points (xo, yo), (xl, yl1) and having the Specified first derivatives 
y0, yatthe points. We can obtain the four coefficients 万 3, 于, 万， Po by solving 


Co) = 矶 如 十 柬 嫌 十 析 xzo 十 硬 = 加 


jc = 柬 好 十 环 好 十 拉 十 硬 =J .6 
jxo0) 一 3Exz 十 2PPxzo 十 责 一 荔 可 


1 一 3 矶 好 十 2PPz 十 王 世 


As an alternative, we approximate the Specified derivatives at the data points by 
their differences 


下 jxX0 十 8) 一 Po)  2 一 0 二 AI) 一 AI 一 上 ) 太一 3 
二 攻 二 assemsseya 
8 8 8 


2 
(3.0.3) 
and find the Lagrange/Newton polynomial matching the four points 


(xzo, yo0)，(z2 一 X0 十 8 六 一 加 十 ys, (03 一 X1 一 8 轨 一 1 一 8)，(Cxl 7y1) 
(3.0.4) 
The MAILAB routine“hermit()”constructs Eq. (3.6.2) and Solves it to get 
the Hermite interpolating polynomial coefficients for a single interval given the 
two end points and the derivatives at them as the input arguments. The next 
routine“hermits()”uses“hermit()”to get the Hermite coefficients for a set 
of multiple Subintervals. 


function H = hermit(x0,y0,dy0,x1,y1,dy1) 
= [x0^3 X0^2 X0 二 X1^3 X1^2 
3*Xx0^2 2x*x0 1 0 XDAE28X1 
[y0 y1 dyo dy1]'; %5Eq.(3.6-2) 
(A\b) ; 


function H = hermits(xy)dy) 
% finds Hermite interpolating polynomials for multiple Subintervals 
%Input : [xy],dy - points and derivatives at the points 
#%Output: H = coefficients of cubic Hermite Interpolating polynomials 
forn= 1:1length(x)-1 

H(n,:) = hermit(0,y(n) ,dy(n) ,xn + 1) -xn)yGn+ 1)dyGn+ 1)) 
end 





卫 xample 3.4. Hermite Interpolating Polynomial. Consider the problem of find- 
ing the polynomial interpolation for the NXW 十 1=4data points 


{(0, 0), (1 1D, (2,4), (3, 5)} (E3.4.1) 
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Subject to the conditions 


jp0(xo) = 一 710(0) =2， 和 (=0， 1p(2) 王 0， PN) 一 13(03) 一 2 
(上 E3.4.2) 
Eor this problem，we only have to type the following statements in the MAT- 
LAB command window. 


>>X [0123];jy= [0145];dy= [2002]; x= [0:0.01:3]; 
>>H = hermits(x,y,dqy); yI = ppval(mkpp(x;,H)，Xxi); 


3.7 TWO-DIMENSIONAL INTERPOLATION 


In this section we deal with only the simplest way of two-dimensional 
interpolation 一 that 1S，a generalization of piecewise linear interpolation called 


























































































































| 加 和 | 甸 甸 甸 攻 My) (ec, 儿 ) (xs 外) 
本 面 面 国 
| 2] 和 | J2 入 以 攻 MyJ2) (ec, 儿 ) (xs, J) 
本 面 面 国 
| 贺 和 | Jo 员 多 攻 MX， Js) (ec, 区) (xs, Jo) 
Y (X, Y) 
= [LI Xe2 和 ] y=[M 多 多 ] [X, Y] = meshgrid (x, y) 
21,1 21,2 21,3 
乙 
22,1 22,2 22,3 
23, 1 23, 2 23, 3 
Zi= interp2 (X,Y,Z, Xi Yi 
orinterp2 (x, y, Z, Xi, Yi) 7 
21,1 21,2 214,3 21,4 21,5 
Z1 己 .2 了 2Z2,4 22,5 
.1 .2 .3 2Z3, 4 23,5 
扩 Zi 1 7 1 1 
24,1 24,2 24,3 24,4 24,5 
五 ,1 五, 石 ,3 厂 4 265,5 
=[ 革 和 个 允 对 基 Yi=[ 加 大 内 和 赔 [Xi Yi = meshgrid (xi yi) 
Xi Yi (Xi Yi) 
对 | 和 准 | 准 | 内 | 站 | 儿 | 基 | 明基 | 攻略, 中 )| 0 Wi) | (04 外) | 06, 攻 ) 
对 | 稚 | 准 | 内 | 咎 | 中 | 只 | 咕 | 笑 | 隆 | | (04 只)| 0, 只)| 0, 只) | 0 只) | (06, 吃 ) 
对 | 稚 | 准 | 内 | 咎 || 猴 | 儿 | 铬 | 詹 | 中 | | 0 欠 ) | 0 多) | 40, 欠 ) | 0 内 ) | 406, 区 ) 
对 | 稚 | 准 | 内 | 站 | 向 | 其] 惧 | 居 | 居 ||0 基 ) | (从 其) | (0 J) | 0, 由) | 4, 中) 
本 辐 骆 厅 卫 思 本 四 | 本 双 硬 赂 本 到 区 双 卫 允 用 本 区 区 RE 7 生 到 
和 | 和 ,| 知 | 作 | 阁 外 里 | 里 | 睹 | 里 | 上 | | 0 上 拖 ) | (和 ,上 ) | 0 折 ) | 0 折 ) | (, 护 ) 



























































Figure 3.8 A two-dimensional interpolation using Zi = interp2() on the grid points 
[Xi,YI] generated by the meshgrid() command. 
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the bilinear interpolation. The bilinear interpolation for a point (x, y) on the rect- 
angular Sub-region having (xm 1, yi -1) and (xm yn) as its leftrupperright-lower 
corner points is described by the following formula. 


充 站 一天 X 一 Xi _1 
z00 1 = 一 21 十 一 zol (3.7.1a) 
X1a 一 7 一 1 X1a 一 7 一 1 
Xi 一 苞 X 一 Xi 1 
z00 = 一 zoln+ 一 oa (3.7.1b) 
X1a 一 7 一 1 X1a 一 7 一 1 
一》 y》 一 yn-1 
zy) = 一 200 -DT 十 一 20 yn) 
1 一 yz 一 1 1. 一 J7 一 1 
] 


一 人 一 X)Om 一 )zm-Ln-l 


十 (x Xm_1)(Oyn 一 y)zmm -1l 十 (xm 二 X)(y 四 Jr1DZoLn 


十 (X “本 Xi_1)(y ES yn_1)zmn]} for Xml 三 X 三 MX， yn 一 1 忒 了》 忒 y7m 
(3.7.2) 


function ZI = intrp2(x,y,Z,XiIyI) 
s%5TO interpolate Z(x,y) on (xiyyi) 
M= length(x); N= length(y); 
MI = length(xi); NI = length(yI); 
for mi = 1:MIi 

for nai 

for 由 


end 
if break1 > 0 break， end 
end 
if break1 > 0 break， end 
end 
end 
end 





This formula is cast into the MAILAB routine “intrp2()”, which is so named 
in order to distinguish it from the MAILAB built-in routine“interp2()”. Note 
that in reference to Fig. 3.8, the given values of data at grid points (x(m) ,y(n) ) 
and the interpolated values for intermediate points (xi(m),yi(n)) are Stored in 
Z(nim) and Zi(n;m), respectively. 
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do_interp2.nm 
2-dimensional interpolation for EX 3.5 
-2:0.1:2; yIi = -2:0.1:2; 
[Xi,Yi]l = meshgrid(xiyyI); 
Z0 = Xi.^2 + Yi.^2; 5%5(E3.5.1) 
Subplot(131)，mesh(Xi,Yi,Z0) 
= -2:0.5:2; yY = -2:0.5:2; 
[X,Y] = meshgrid(x,y); 
Z=X.^2+Y.^2i 
Subplot(132)，mesh(X,Y,Z) 
ZL = interp2(x,y,Z,Xi,Yi); %built-in routine 
Subplot(133)，mesh(xi,yi)Zi) 
ZLi = intrp2(x,y,Z,Xxiy,yi); %our own routine 
pause，mesh(xi,yIyZi) 
norm(Z0 - Zi)Vnorm(Z0) 


D 
26 
26 





上 Example 3.$S， Two-Dimensional Bilinear Interpolation. We consider interpolat- 
ing the Sample values of a function 


Fo, 人 二 妇 十 六 (E3.5.1) 


for the 3 x3 grid over the 21 x 21 grid on the domain 刀 ={(c,y)| 一 2 和 xx 芭 
2, -2<》< 97. 

We make the MAILAB program“do_interp2.m"，which uses the routine 
“intrp2()”to do this job，compares its function with that of the MAILAB 
built-in routine“interp2()”, and computes a kind of relative error to estimate 
how close the interpolated values are to the original values. The graphic results 
of running this program are depicted in Fig. 3.9，which shows that we obtained 
a Ieasonable approximation with the error of 2.0% from less than 1/16 of the 
original data. It is implied that the Sampling may be a Simple data compression 
method, as long as the interpolated data are little impaired. 


3.8 CURVE FITTING 


When many sample data pairs {(Cxk, yi), 大王 0: Mare available, we often need 
to grasp the relationship between the two variables or to describe the trend of the 





2 二 -2 -2 


(a) True function (b) The function over (c) Bilinear interpolation 
sample grid 


Figure 3.9 Two-dimensional interpolation (Example 3.5). 
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data, hopefully in a form of function y = jx). But as mentioned in Remark 3.1， 
the polynomial approach meets with the polynomial wiggle and/or Runge phe- 
nomenon，which makes it not attractive for approximation purpose. Although the 
cubic spline approach may be a roundabout toward the Smoothness as explained 
in Section 3.3$, it has too many parameters and So does not Seem to be an ef 但- 
client way of describing the relationship or the trend，since every Subinterval 
needs four coeffcients. What other choices do we have7 Noting that many data 
are Susceptible to some error，we don't have to try to find a function passing 
exactly through every point. Instead of pursuing the exact matching at every data 
point，we look for an approximate function (not necessarily a polynomial) that 
describes the data points as a whole with the smallest error in Some Sense, which 
jls called the curve fitting. 

As a reasonable means，we conslder the least-squares (LS) approach to min- 
imizing the sum of Squared errors，where the error ls described by the vertical 
distance to the curve from the data points. We will look over various types of 
fitting functions in this section. 


3.8.1 _ Straight Line Fit: A Polynomial Function of First Degree 


Ithere ls Some theoretical baslis on which we believe the relationship between 
the two variables to be 
0x 十 9 一》 (3.8.1) 


we Should set up the following System of equations from the collection of many 
experimental data: 


OIX1 十 00 三 7 
Ox2 十 00 三 六 
OXMN 十 00 王 yw 
了 y1 
40=-y win4=|2 | | y= | 呈 | G.82) 
XZM 1 yM 


Noting that this apparently corresponds to the overdetermined case mentioned 
in Section 2.1.3，we resort to the least-squares (LS) solution (2.1.10) 


Oo 
bo 一 [ 风 = [474]-147y (3.8.3) 
0 


which minimizes the objective function 


J=lel =140 一 y 必 = [40 一 中 [40 一 习 (3.8.4) 
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Sometimes we have the information about the error bounds of the data, and it ls 
Ieasonable to differentiate the data by weighing more/less each one according to 
its accuracy/reliabijlity. This policy can be Implemented by the weighted least- 
Squares (WLS) solution 


O 
b9 一 9 [4T7WA4-LATWy (3.8.5) 
which minimizes the welighted objective function 


jw = [40 一 了 [40 一 杂 (3.8.6) 


If the weighting matrix is 砚 = YL 一 R-TR-I， then we can write the WLS 
Solution (3.8.5) as 





bw 一 [| 一 [(R-I4)7CR-I4)]-ICR-I4)7R-Iy 一 [474R] 1 AAYR 
凡 0 
(3.8.7) 
where 


4R 一 R-A4， yR 一 Rr-Iy， 机 =V-L =R-TR-l (3.8.8) 


One may use the MAILAB built-in routine “lscov(A,y,V)”to obtain this 
WLS solution . 


3.8.2 ”Polynomial Curve Fit: A Polynomial Function of Higher Degree 


Hthere ls no reason to limit the degree of ftting polynomial to one, then we may 
increase the degree of fitting polynomial to, say，N in expectation of decreasing 
the error. Still, we can use Eq. (3.8.4) or (3.8.6), but with different definitions of 


4and0 as 六 
区 Xl 1 ON 
N 
4=-| 妈 并， 6= (3.8.9) 
ES 0 
:Xi 1 oo 


The MAILAB routine“polyfits()”Pperforms the WLS or LS Scheme to 
fnd the coefficients of a polynomial fitting a given Set of data points, depending 
on whether or not a vector (P) having the diagonal elements of the weighting 
matrix 信 ls given as the fourth or fifth input argument. Note that in the case of 
a diagonal weighting matrix 三 , the WLS solution conforms to the LS solution 
with each row of the information matrix 4 and the data vector y multiplied by 
the corresponding element of the weighting matrix 厂 . Let us See the following 
examples for its usage: 
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function [th,err,yi]l = polyfits(x,y,N,Xir) 
%5Xx,y : the row vectors of data pairs 
: the order of polynomial(>=0) 
: reverse Weighting factor array of the same dimension as y 
Length(x); X= Xx(:);jyY=y(:); %Make all column vectors 
if nargin == 4 
if length(xi) == M P= xi;i xi= Xj %With input argument (xy;Nr) 
elser= 1; %With input argument (xy;N,XI) 
end 
elseif nargin == 3，Xi ; 点 With input argument (xy;N) 
end 
A(:,N + 1) = 
forn=N:-1: 
if length(r) 
form=1: 
end 
th 
ye 
VyI 


关 1) 1; 

Al(:,n) = A(:,n+1).*xj end %Eq.(3.8.9) 
M 
( 


one 
1 


M，Alm,:) = Alm,:)/rm);y(m) = y(m)yVrm);i end %5Eq.(3.8.8) 
(A\y)” %5Eq.(3.8.3) or (3.8.7) 

polyval(th,x); err = norm(y - ye)/norm(y); %estimated yY Values，error 
polyval(th,xI) ; 


%5do_polLyTfit 
1oad xy1.dat 
X= xyl1(:,1);yY= xy1(:，2); 
[x,I] = Sort(x); y = y(I); %sort the data for plotting 
xi= min(x)+[0:100]/100*(max(x) - min(x)); %intermediate points 
for L = 1:4 
[th,err,yi]l = polyfits(X,yY,2x*1 - 1,Xi); err 5%5LS 
Subp1lot(220+1) 
plot(x,y，Kk* ,Xi yi b: ) 


%5Xy1.dat 
-3.0 





卫 xXample 3.0. Polynomial Curve Fit by LS (Least Squares). Suppose we have 
an ASCII data file“xy1.dat”containing a set of data pairs {(xck， yz 一 0:0} in 
two columns and we must fit these data into polynomials of degree 1, 3, 9, and 7. 


人 0.8958 | 一 1.5651 | 3.4565 | 3.0601 | 4.8568 | 3.8982 





We make the MAILAB program“do_polyfit.m"，which uses the routine 
“polyfits()”to do this job and plot the results together with the given data 
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6 上 一 6 上 刁 
4 之 4 
2 上 刁 2 上 刁 
米 米 
0 上 二 0 上 刁 
_2 上 上 米 了 -2 米 J 
一 4 一 4 | | 
-4 -2 0 2 4 -4 -2 0 2 4 
(a) Polynomial of degree 1 (b) Polynomial of degree 3 
8 8 
6 上 导 6 上 刁 
4 世 4 
上 罗 本 国 - 
2 上 一 2 上 刁 
洲 
0 上 于 0 上 刁 
_2 上 上 米 了 了] -2 过 
_4 | | | | | | | 
-4 -2 0 2 4 -4 -2 0 2 4 
(c) Polynomial of degree 5 (d) Polynomial of degree 7 


Figure 3.10 Polynomial curve fitting by the LS (Least-Squares) method. 


points as depicted in Fig. 3.10. We can observe the polynomial wiggle that the 
OoScillation of the fitting curve between the data points becomes more pronounced 
with higher degree. 


了 kxample 3.7. Curve Fitting by WLS (Weighted Least Squares). Most experimen- 
tal data have Some absolute and/or relative error bounds that are not uniform for 
all data. 开 we know the error bounds for each data，we may give each data a 
welght inversely proportional to the Slze of its error bound when extracting valu- 
able information from the data. The WLS9S solution (3.8.7) enables us to reflect such 
a weighting Strategy on estimating data trends. Consider the following two cases. 


(a) Suppose there are two gauges A and B with the same function, but dif- 
ferent absolute error bounds 士 0.2 and 士 1.0, respectively. We used them 
to get the input-output data pair (Xwym) as 


{(1 0.0831), (3, 0.9290),， (5, 2.4932),， (7, 4.9292), (9, 7.96005)} 
from gauge A 


{(2, 0.9536), (4, 2.4836),， (0, 3.4173),， (8, 0.3903),， (10, 10.2443)} 
from gauge B 


Let the fitting function be a second-degree polynomial function 


?》 一 ax2 十 ax 十 a0 (上 3.7.1) 
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10 
10 上 上 刁 
8 了 
8 了 
6 了 
6 了 
linearl 
性 了 4 yY 昌 
4 linearly LS | interpolated 
interpolated data 
data g 区 
2 刁 栈 K 了 
2|WLs 
8 | | | | | 0 | | | 
0 2 4 6 8 10 0 5 10 15 20 


al) Fitting to a polynomial y= azX2 + aiX+ 耻 
9 y Y=a2 1 0 


(b 


mA 


(b) Fitting to y= ax 


Figure 3.11 LS curve fitting and WLS curve fitting for Example 3.7. 


To fnd the parameters az, al,，and da0，we write the MAILAB program 
“do_wlse1.m”, which uses the routine“polyfits()”twice, once without 
weighting coefficients and once with weighting coefficients. The results 
are depicted in Fig. 3.11a，wphich shows that the WLS curve fitting tries 
to be closer to the data points with smaller error bound，while the LS 
curve fitting weights all data points equally，which may result in larger 
deviations from data points with small error bounds. 

Suppose we use one gauge that has relative error bound 士 40[%] for 
Imeasuring the output y for the input values x 一 [1 3, 93, ...，19] and so 
the Size of error bound of each output data is proportional to the masgnitude 
of the output, We used it to get the input-output data palr (xmym) as 


{(1,4.7334), (3, 2.1873), (5, 3.00607),， (7, 1.4273), (9, 1.7787) 
(11, 1.2301), (13, 1.6052), (15, 1.35353), (17, 1.3985), (19, 2.0211)} 


Let the fitting function be an exponential function 


》 了 一 QX (上 E3.7.2) 
To find the parameters a and D，we make the MAILILAB program 
“do_wlse2.m", which uses the routine “curve_fit()”without the weight- 
jing coefficients one time and with the weighting coeffcients another time. 
The results depicted in Fig. 3.11b shows that the WLS curve fitting tries to 
get closer to the data points with Smaller |y|, while the LS curve fitting pays 
equal respect to all data points, which may result in larger deviation from 
data points with small |y|. Note that the MAILAB routine “curve_fit()” 
appears in Problem 3.11,， which Implements all of the Schemes listed in 
Table 3.5 with the LS/WLS Solution. 
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(cf) Note that the objective of the WLS scheme is to put greater emphasis on more 
reliable data. 


%5do_wW1lse1 for EX.3.7 
Clear，Clf 
xx= [13579246810]; %input data 
y = [0.0831 0.9290 2.4932 4.9292 7.9605 ..， 

0.9536 2.4836 3.4173 6.3903 10.2443]; %output data 
eb = [0.2*ones(5,1); ones(5,1)]; %error bound for each y 
[x,I] = Sort(x);yY =y(i);i eb = eb(1I); %sort the data for plLotting 
errorbar(x,y,eb，:')，hold on 
N = 2; %the degree of the approximate polynomial 
xi = [0:100]/10; %interpolation points 
[thl,errl,y1I] = polyfits(x,y,N,XxI) ; 
[thwl,errw1l,ywl1] = polyfits(x,y;,N,Xxiyeb); 
plLot(xi;y1L，b' ，xiywl， mr ) 
%KC = 0; thlc = Curve_fit(x,y，KC,N,Xi); %5for cross-check 
s%thwlc = curve _fit(x,y，KC,N,Xiyeb); 


%5do_W1Sse2 

Clear，Clf 

x= [1:2:20]; Nx = Jength(x); %changing Input 

xi = [1:200]/10; %interpolation points 

eb = 0.4*ones(Size(X)); #%serror bound for each y 

y = [4.7334 2.1873 3.0067 1.4273 1.7787 1.2301 1.6052 1.5353 .,， 
1.3985 2.0211] ; 

[x,I] = Sort(x);yY =y(i); eb = eb(1I); %sort the data for plLotting 

eby = y.*xeb; %our estimation of error bounds 

KC = 6j [thlcerr,y1l] = curve_fit(xy，KC,0,Xi) 

[thwJlcerr,ywlI] = curve_fit(x,y,KC,0,Xxiyveby) 

errorbar(x,yyeby)，hold on 

plLot(xi;,y1L，b' ，xiywl， mr) 





3.8.3 Exponential Curve Fit and Other Functions 


Why don't we use functions other than the polynomial function as a candidate 
for fitting functions? There is no Teason why we have to Stick to the polynomial 
function, as illustrated in Example 3.7(b). In this section，we conslder the case 
in which the data distribution or the theoretical background behind the data tells 
us that it ls appropriate to fit the data into some nonpolynomial function， 
Suppose it ls desired to fit the data into the following exponential function， 


Eee 二 池 (3.8.10) 
Taking the natural logarithm of both Sides，we linearize this as 


4aX 十 nc 一 Iny (3.8.11) 
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Table 3.5 ”Linearization of Nonlinear Functions by Parameter/Data Transformation 


Function to Fit 





(CD y= 荆 十 
了 

O ?= 二 

G) =a 信 

(人 ?=Bee 


(3) 7 一 C 一 De 


(D 一 az 
(7) 一 ax e 姑 
8) y = 一 一 一 
(8) 》 Te 


(0, 0)0,C = y(co)) 
(9) > 一 alnx 十 


Linearized Function 


1 
yy 一 0 一 十 一 》=ax' 十 D 
区 


1 


一 一 KK 十 一 一 多 王 CQX 十 以 


1 

> 忆 
Iny=(np)x 二 ina 
过 区 一 QX 十 以 


ny 一 ax 十 np 一 交 一 az 十 


ln(C 一 ) = 一 ax 十 In2 
一 多 一 GX 十 以 
Iny 一 pdnx) 十 ina 
一 一 CX 十 已 
ny 一 Inx=px 十 na 
一 多 =CX 十 以 
mn 人 (和 -可 =e+ne 
了 

一 多 一 ax 十 已 

一 yy 一 QX' 十 尺 


Variable Substitution/ 
Parameter Restoration 


< 1 > 1 
王 


多 一 Iny, =e7 
7 =In(C 一切 
Q 一 一 0,D 一 ce 

六 一 Inyx 一 Inx 


Q 一 e2D 一 dl 


y = In(y/z) 


So that the LS algorithm (3.8.3) can be applied to estimate the parameters ad and 
lnc based on the data pairs {(Cxzz, In y) ,上 一 0: AI)}. 

Like this, there are many other nonlinear relations that can be linearized to fit 
the LS algorithm, as listed in Table 3.5. This makes us believe in the extensivVe 
applicability of the LS algorithm. 开 you are interested in making a MAILAB 
Ioutine that Implements what are listed in this table, see Problem 3.11, which lets 
you try the MAILAB built-in function “lsqcurvefit(f,th0,xiy)”that enables 
one to use any type of function (f) for curve fitting， 


3.9 FOURIER TRANSFORM 


Most signals existent in this world contain various frequency components，where 
raplidly/slowly changing one contains high/low-frequency components. Fourier 
Series/transform ils a_ mathematical tool that can be used to analyze the fre- 
quency characteristic of periodic/aperiodic Signals. There are four similar defini- 
tions of Fourier Series/transform, namely, continuous-time Fourier series (CtFS)， 
continuous-time Fourier transform (CtET)，discrete-time Fourier transform 
(DtFET)，and discrete Fourier Series/transform (DEFESADEFT). Among these tools， 
DFT can easily and efficiently be programmed in compnuter languages and that's 
why we deal with just DFT in thlis section. 
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Suppose a sequence of data {x[z] 一 XY(OT) ,由 一 0:M 一 HGCT:the Sampling 
period) ls obtained by Sampling a continuous-time/sSpace Signal once every 了 
Seconds. The NMW(> MI)-point DFTVIDFT (inverse DFT) pair is deftined as 


N 一 1 
DFT: 和 (oO = 》 xm]e 72 ， KK=0:N-1 (3.9.1a) 
1 一 0 
1 N 一 1 
IDEFT: x[m] = 3 和 (人 Je72rnK/N ， 1=0:NW-1 (03.9.1b) 
大 一 0 


了 R 了 emark 3.3. DFSDEFT (Discrete Fourier Series/Transformy) 


0. Note that the indices of the DFTVIDFT sequences appearing in MAILAB 


1. 


3.9.1 


range from 1 to N. 


Generally, the DFT coefficient X(K) is complex-valued and denotes the 
masnitude and phase of the Signal component having the digital frequency 
4 人 一 KS20 一 27K/N[Iradl, which corresponds to the analog frequency ck 一 
Ko 一 K20/ 了 一 27KANT[Irad 人 s]. We call 420 一 27r/N and oo 王 27r/NT 
(N represents the Size of DFT) the digital/analog fundamental or resolution 
frequency, Since it ls the minimum digital/analog fredquency difference that 
can be distinguished by the NW-point DFT. 


. The DFS and the DFT are essentially the Same, but different in the range 


of time/frequency interval. More Specifically，a Signal x[2] and its DFT 
X(K) are of fnite duration over the time/frequency range{0O0<72<N 一 1 
and {0 <K< N 一 1j, respectively, while a signal X[2] (to be analyzed by 
DFS) and its DFS 驻 (KE) are periodic with the period N over the whole set 
of integerSs. 


. FFT (fast Fourier transform) means the computationally efficient algorithm 


developed by exploiting the periodicity and symmetry in the multiplying 
factor er to reduce the number of complex number multiplications 
from NM2 to (N/2) log” NM (CN represents the Size of DFT). The MAILAB 
built-in functions “fft() /ifft()”implement the FEFEIVIEFEFT algorithm for 
the data of length N = 2 (1 represents a nonnegative integen). If the length 
Mof the original data Sequence is not a power of 2, it can be extended by 
padding thetail part ofthe sequence with zeros, which is called zero-padding. 


FFT Versus DFT 


As mentioned in item 3 of Remark 3.3,， FFTVIEEFT (inverse FFT) is the compu- 
tationally efficient algorithm for computing the DFIVIDFT and is fabricated into 
the MAILAB functions“fft() /ifft()”. In order to Practice the use of the 
MAILAB functions and realize the computational advantage of FFIJVIFFT over 
DFTIVIDFT, we make the MAILAB program “compare_dft_fft.m”. Readers are 
Iecommended to run this program and compare the execution times consumed by 
the 1024-point DFTVIDEFT computation and its FFIVIFEFT scheme, seeing that the 
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Iesulting Spectra are exactly the Same and thus are overlapped onto each other 
as depicted in Fig. 3.12. 


s%5Ccompare_DFT_FFT 
Clear，Clf 
2 人 10; nn= [oO:N - 1]) 
X = CoSsS(2*pi*200/Nx*n)+ 0.5x*Sin(2*pix300/Nxn) ; 
七 IC 
for kK= 0:N - 1，X(k+1) = Xx*exp(-j*2x*pix*kx*n/N). ; end %DFT 
k= [O:N - 1]; 
forn=0O:N- 1，xrn+ 1) = Xx*exp(j*2x*pix*kx*n/N). ; end %IDFT 


time_dft = toc %number of floating-point operations 
plot(k,abs(X))，pause，hold on 

七 IC 

X1 = fft(X); %FFT 

xrl1 = ifft(X1); %IFFT 

time_fft = toc %number of floating-point operations 
clf，plot(k,abs(X1)，Pr' ) smagnitude Spectrum in Fig， 





3.9.2 Physical Meaning of DFT 


In order to understand the physical meaning of FFT，we make the MAILAB 
program “do_fft”and run it to get Fig. 3.13, which shows the magnitude Spectra 
of the Sampled data taken every 7 seconds from a two-tone analog Signal 


X(1) 一 Sin(1.Srr1) 十 0.9cos(3 增 人 (3.9.2) 


Readers are recommended to complete the part of this program to get Fig. 3.13c,d 
and run the program to see the plotting results (See Problem 3.10). 

What information do the four Spectra for the same analog Signal x(1) carry? 
The magnitude of X.(K) (Fig. 3.13a) is large at 上 = 三 2 and 9, each corresponding 
to Kowo 一 27rK/NT 一 27rK/3.2 王 1.23 久 1.57r and 3.1237 久 3 六 . The magni- 
tude of Xp([) (Fig. 3.13b) ls also large atK 一 2 and 9$,each corresponding to 
Ko 一 1.23 六 久 1.37r and 3.123r 久 37. The magnitude of Xe(K) (Fig. 3.13c) ls 
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Figure 3.12 The DFTFFD {XO,k=0:N-1 of x[N = cos(C2r x200n/N) 十 0.5sin 
(2r x 300n/N) forn = 0:N-1N= 210 = 1024). 
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%qdo_ fft (to get Fig. 3.13) 
CJear，CLf 
晤 二 证 2 .1L W2=3x*pIj %two tones 
32;jn= [O:IN- 1];T=0.1; %5sampling period 

nxT; Xxan = Sin(w1x*t) + 0.5*SIin(W2*t) 
Subplot(421)，Sstem(t,xan，.) 
k=O0O:N- 1; Xa=TfTft(xan); 
dscrp=norm(xan-real(ifft(Xa))) %5x[n] reconstructible from IFFT{X(k)}? 
Subplot(423)，Sstem(k,abs(Xa)，. ) 
%upsampJling 
N=64;jn= [0O:N- 1];T=0.05; %sampling period 
tt = nx*T; xbn = Sin(w1xt)+ 0.5*Sin(W2x*t) ; 
Subplot(422)，Sstem(t,xbn，.) 

= 0O:N- 1; xb=TfTft(xbn); 
Subplot(424)，Sstem(k,abs(xb)，. ) 
%zero-padding 

= 64;n= [0O:N-1];T=0.1; %sampling period 
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Figure 3.13 DFT spectra of a two-tone signal. 
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large at 上 = 4,3 and 9,10, and they can be alleged to represent two tones of Koo 一 
27K/N7T =27rK/0.4 久 1.237 ~ 1.30237 and 2.81257 ~ 3.123T. The magni- 
tude of Xy(K) (Fig. 3.13d) is also large at 大 一 9 and 10, each corresponding to 
Kc0 一 1.3023 元 久 1.3 and3.125 区 久 37. 

Itis strange and interesting that wehavemany differentDEFT spectra forthe same 
analog Signal, depending on the DFT size, the Sampling period, the whole interval， 
and zero-padding. Compared with Spectrum (a), Spectrum (b) obtained by decreas- 
ing the Sampling period 7 from 0.1s to 0.053s has wider analog frequency range 
[0,2r/Z],butthe same analogresolution frequency is oo 一 4020/ 有 取 =27r/No7 一 
/1.0 三 27r/Na7a; consequently, it does not present us with any new information 
over (a) forall increasednumber of data points. The shorter sampling period may be 
helpful in casethe analog signal has some spectral contents of frequency higherthan 
T/ 7. The Spectrum (c) obtained by zero-padding has a better-looking, Smoother 
Shape, but the vividness ls not much Improved compared with (a) or (bj, Since the 
Zeros esSsSentially have no valuable information in the time domain. In contrast with 
(b) and (c), Spectrum (d) obtained by extending the whole time interval shows us 
the Spectral information more distinctly. 

Note the following things: 


。 Zero-padding in the time domain yields the interpolation (Smoothing) effect 
in the frequency domain and vice versa, which will be made use of for data 
Smoothing in the next Section (See Problem 3.19). 

H 芝 asignal is of fnite duration and has the value of zeros outside its domain 
on the time axis，its Spectrum is not discrete，but continuous along the 
frequency axis，while the Spectrum of a periodic Signal ls discrete as can be 
Seen in Fig. 3.12 or 3.13. 

The DFT values X(0) andX(N/2) represent the Spectra of the dc component 
(So =0) and the virtually highest digital frequency components (42N/2 一 
N/2x27/N = 六 [rad]), respectively， 


Here, wehave Something questionable. The DFT Spectrum depicted in Fig. 3.12 
Showsgs clearly the digital frequency components 42200 一 27 X 200/N and 423o0 一 
2r x 300/N[rad](CN = 210 = 1024) contained in the discrete-time signal 


X[z] = cos(2r x 200n/N) 十 0.5sin(C2zr x 3000/N)， N=20 = 1024 

(3.9.3) 
and SO we can find the analog frequency components or 一 SCK/T as long as 
the Sampling period 7 js known，while the DFT spectra depicted in Fig. 3.13 
are So unclear that we cannot discern even the prominent frequency contents. 
What's wrong with these Spectra7 It is never a “Tight-or-wrong” problem. The 
only difference is that the digital frequencies contained in the discrete-time Signal 
described by Eq. (3.9.3) are multiples of the fundamental frequency S2o 一 27r/N， 
but the analog frequencies contained in the continuous-time Signal described by 
Eq. (3.9.2) are not multiples of the fundamental frequency oo 一 2r/NT7; hm 
other words, the whole time interval [0, V7) is not a multiple of the period of 
each frequency to be detected. The phenomenon wherepy the Spectrum becomes 
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blurred like this is said to be the“ “leakage problem ”. The leakage problem occurs 
in most cases because we cannot determine the length of the whole time interval 
in Such a way that it is a multiple of the period of the signal as long as we don't 
know in advance the frequency contents of the Signal. 玉 we knew the frequency 
contents of a Signal, why do we bother to find its Spectrum that is already known? 
As a measure to alleviate the leakage problem, there ls a windowing technique 
[O-1, Section 11.2]. Interested readers can See Problem 3.18. 

Also note that the periodicity with period Nthe DFT size) of the DFT 
Sequence X(K) as well as X[1],，as can be manifested by substituting 上 十 1N 
(1 Tepresents any integer) for 上 in Eq. (3.9.1a) and also Substituting 刀 十 MIN 
for 7 in Eq. (3.9.1b). A real-world example reminding us of the periodicity of 
DFT Spectrum is the So-called stroboscopic effect whereby the wheel of a car- 
Tiage driven by a horse in the Scene of a western movie looks like spinning at 
lower Speed than its real Speed or even in the reverse direction. The periodicity 
of x[z] is Surprising,，pbecause we cannot imagine that every discrete-time Slgnal 
is periodic with the period of NW，which is the variable Size of the DFT to be 
determined by us. As a_ matter of fact, the“weird” periodicity of x[z] can be 
Iegarded as a kind of cost that we have to pay for computing the sampled DEFT 
Spectrum instead of the continuous Spectrum X (w) for a _ continuous-time Signal 
X(f)，which is originally defined as 


X(w) = 1 xDe-yor di (3.9.4) 


Oo 


Actually, this ls to blame for the blurred spectra of the twor-tone Signal depicted 
in Fig. 3.13. 


3.9.3 Interpolation by Using DFS 


function [xi,Xi]l = interpolation_ by_DFS(T,x,Ws，ti) 
%5T : Sampling interval (sample period) 
%5Xx : discrete-time Sedquence 
%WS: normalized stop frequency (1.0=pi[rad]) 
%ti: interpolation time range or # of divisions for 工 
if nargin < 4，ti = 5; end 
if nargin <3 |Ws>1，Ws= 1; end 

= Jength(x) ; 
if length(ti) == 1 

ti = 0:T/ti:(N-1)*T; %Subinterval divided by ti 
end 
ks = ceil(Ws*N/2) ; 
Xi = fft(x); 
Xi(ks + 2:N - ks) = zeros(1;N - 2*ks - 1); %filtered Spectrum 
xi = zeros(1,Length(ti) ); 
for k = 2:N/2 

Xli = Xi+Xi(k)*exp(jx2*pix(k - 1)*tz/N/T) ; 

end 
xi = real(2*Xxi+Xi(1)+Xi(N/2+1)*xcos(pixti/T))/VN; %Eq.(3.9.5) 
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s%interpolate_by_DFS 

Clear，Clf 

w1 = pij w2 = .5*pij %two tones 

N=32;n= [0O:N- 1];T=0.1; tt= nx*Ti 
Sin(w1x*t)+0.5*Sin(Ww2x*t)+(rand(1;,N) - 0.5); %0.2*xSin(20* 七 ) ; 

ti= [0:T/5:(N - 1)*T]; 

Subplot(411)，plot(t,x，k. ) %original data Sequence 

title('original seduence and interpolated Signal ) 

[xi,Xi]l = interpolation_by_DFS(T,x,1，,tz); 

hold on，plot(ti,xi，r') s#%sreconstructed Signal 

k= [0O:N - 1]; 

Subplot(412)，stem(k,abs(Xi)，k.') #%original Spectrum 

title('original Spectrum ') 

[xi,Xi]l = interpolation_by_DFS(T,x,1/2,ti); 

Subplot(413)，stem(k,abs(Xi)，r,.') #%filtered Spectrum 

tit1le('filtered Spectrum ') 

Subp1lot(414)，plot(t,Xx，k. ，ti xi r' ) s%5filtered Signal 

title('filtered/smoothed Signal ') 





We can use the DFS/DEFT to interpolate a given sequence XY[1] that 1$ Supposed 
to have been obtained by sampling Some Signal at equidistant points (instants). 
The procedure consists of two Steps; to take the NW-point FFT X(C) of x[2] and 
to use the formula 


1 恤 
和 (1 2 》， 和 (Je72rLAT 


|k<N/2 
1 N/2 一 1 
一 六 人 (D) 十 2 》， Real{X(eez2r4 人 7 十 和 CN/2) cos(rL/T)} (3.9.5) 
K 一 1 


This formula ls cast into the routine“interpolation_by_dfs”, which makes 
it possible to filter out the high-frequency portion over (Ws.r，(2-Ws)r) with 
Ws given as the third input argument. The horizontal (tme) range over which 
you want to interpolate the Sequence can be given as the fourth input argument 
ti. We make the MAILAB program“interpolate_by_dfs”, which applies the 
Ioutine to interpolate a set of data obtained by sampling at equidistant points 
along the Spatial or temporal axis and run it to get Fig. 3.14. Figure 3.14a Shows 
a data Sequence Xx[71] of length NW = 32 and its interpolation (reconstruction) 
X(f) from the 32-point DFSDFT X(K) (Fig. 3.14b), while Figs. 3.14c and 3.14d 
Show the (zero-padded) DFT Spectrum X“(K) with the digital frequency contents 
higher than T/2[rad](W/4 < 上 < 3N/4) removed and a Smoothed interpolation 
(fitting curve) x“ (1) obtained from Xe), respectively. This can be viewed as the 
Smoothing effect in the time domain by zero-padding in the frequency domain， 
in duality with the Smoothing effect in the frequency domain by zero-padding in 
the time domain，which was observed in Fig. 3.13c. 
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(c) The spectrum X“(K) of the filtered signal x“( 四 
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Figure 3.14 Interpolation/smoothing by using DFS/VDFT. 


PROBLEMS 


3.1 Quadratic Interpolation: Lagrange Polynomial and Newton Polynomial 


(a) The Second-degree Lagrange polynomial matching the three points 
(xzo，1)，(xc 万 )，and (xz2, 户 ) can be written by Substituting NW 一 2 
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into Eq. (3.1.3) as 


PC = 》 记 P2m(o) = 于 记 1 医 -一 二 (P3.1.1) 


mm=0 Km 4 


Check if the zero of the derivative of this polynomial 一 that is, the root 
of the equation 5 (xY) 三 0 一 1s found as 


， (人 一 X1) 十 (一 22) (人 一 22) 十 (一 2X0) 
1 人 人 
2 二 办 (Cxo 一 X1)(xo 一 X2) 下 Cl 一 X2)(GX1 一 X0) 
(一 X0) 十 (一 XI) 
上 G@c 一 X0)(x2 一 X1) 
JJ0Cx 一 X1 一 32)(xz2 一 21) 十 ee 一 3X2 一 X0)(xzo 一 22) 
十 .PCx 一 X0o 一 XI)GI 一 Xo0) 一 


3 帮 (c 一 xz 十 万 0 一 旭 ) 十 户 Ccd 一 2X1) (P3.1.2) 
， 2{jJGxl 一 32) 十 万 Co 一 ZX0) 十 户 (xo 一 2)} 


You can use the Symbolic computation capability of MAILAB by 
typing the following statements into the MAILAB command window: 


>>Syms X X1 Xx2 Xx3 f0O fl f2 

>>L2 = fO*(X - X1)*(X - X2)/(xX0 - X1)/(xX0 - X2)+..， 
fl1*(X - X2)*(X - X0O)/(x1 - X2)/(x1 - X0)+..， 
f2*(X - XO)*(X - X1)/(x2 - X0)/1(x2 - X1) 

>>pretty(Solve(diff(L2))) 


(b) The Second-degree Newton polynomial matching the three points 
(xzo, jj)， GD 万 ), and (xz2， 户 ) is Eq. (3.2.4). 





12(X) 一 00 十 QI 一 X0) 十 (一 X0)G 一 2X1) (P3.1.3) 
Where 
= 万 ， dl = 了 D 广 = 矿 二 加 
| 一 0 
万 一 万 态 一刀 
万 万 一 也 和 至 
区 二 间 则 克 上 全 区 = 二 (P3.1 朋 
X2 一 X0 X2 一 X0 


Find the zero of the derivative of thlis polynomial. 

(c) From Eq. (P3.1.1) with xo = 一 一 1 xl1 一 0, and xz 一 1, fnd the coef6- 
cients of Lagrange coeffticient polynomials 乙 2 0(x), 2 1(xz), and 元? 2(X). 
You had better make use of the routine “lagranp()”for this job. 


3.2 


3.3 
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(dJ) From the third-degree Lagrange polynomial matching the four points 
(Co 丰 )，G 万 )，(z 户 )，and (3 户 ) with Xo 三 一 3,X1 三 一 2, 22 一 
一 1], and x3 一 0, fnd the coefficients of Lagrange coefficient polyno- 
mials 上 3.0(x), 了 31(0), Za32(X)，and 也 33(x). You had better make use 
of the routine“lagranp()”for this job. 


Error Analysis of Interpolation Polynomial 


Consider the error between a true (unknown) function jx) and the interpo- 
lation polynomial Pv(x) of degree N for some (CN 十 1 points of y = 三 (xz)， 
that ils， 


{(Cxo, yo0),， (xl 7) CN YN)} 
where jx is up to (CN 十 1)th-order differentiable. Noting that the error is 
also a function of x and becomes zero at the (CN 十 了) points，we can write 
it as 
e0 = JPo=CG-xzo0G 一 2 C 一 xzw8c)  (P3.2.0 


Technically，we define an auxiliary function w(1) with respect to 1 as 


VD = DOD 一 PNODO 一 C 一 XU 一 2) LE 一 ZN)SCC) (3.2. 人 





Then, this function has the value of zero atthe (NW 十 2) points!t 一 X0,X1,...， 
XN,X and the 1/2/.…/CN 二 JJ)th-order derivative has (NT 十 TAN /JI 
Zeros, respectively. For l 一 加 such that w+D(10) = 0, we have 


wwr+rDdo = fw+r0o0--0-(V+lDleC) = 0; 


8Cr) 一 w+D(00) (P3.2.3) 


(CV 十 JI! 


Based on this, show that the error function can be Tewriitten as 


的 = JO 一 本 人 =GiG 苛 一 OHDO) 
三 NAX) 二 0 1 N CT 0 
(P3.2.4) 
The Approximation of a Cosine Function 


In the way Suggested below, fnd an approximate polynomial of degree 4 
for 


一 (CCc) 一 cosX (P3.3.1) 


(a) Find the Lagrange/Newton polynomial of degree 4 matching the fol- 
lowing fve points and plot the resulting polynomial together with the 
true function cos X over [一 元 , 十 元 ]. 
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天 0 1 2 3 4 
世 一 元 三 邢 /2 0 十 /2 十 元 
Joo 一 1! 0 1 0 一 1! 


(b) Find the Lagrange/Newton polynomilal of degree 4 matching the fol- 
lowing five points and plot the resulting polynomial on the same graph 
that has the result of (al). 


大 0 1 2 3 4 
XK 六 cosS(9T/10) 立 cos(7Tr/10) 0 区 cos(3r/10) 元 cos(Tr/10) 
J(xb) 一 0.9882 一 0.2723 1 一 0.2723 一 0.9882 
(c) Find the Chebyshev polynomial of degree 4 for cos x Over [一 元 , 十 元 ] 
and plot the resulting polynomial on the same graph that has the result 
of (a) and (b). 
Chebyshev Nodes 


The current Speed/pressure of the liquid Howing in the pipe, which has irreg- 
ular radius,， will be different from Place to place. If you are to install Seven 
Speed/pressure gauges through the pipe of length 4 m as depicted in Fig. 
P3.4, how would you determine the positions of the gauges so that the max- 
imum error of estitmating the Speed/pressure over the interval [0, 4] can 
be minimized? 




















上 一 | | r 二 > X 
0 1 2 3 4 
Figure P3.4 Chebyshev nodes. 
Pade Approximation 
For the Laplace transform 
下 (G) 一 ee (P3.5.1) 


Tepresenting the delay of 7 [seconds]，we can write its Maclaurin Series 
expansion up to fifth order as 


和 


Mic(s) 人 兰 1 一 87 十 本 相 机 机 了 














(P3.5.2) 


(a) Show that we can Solve Eq. (3.4.4) and use Eq. (3.4.1) to get the Pade 
approximation as 
40 十 419 ”1 工 一 人/2s 


人 人 三 人 一 了 8 
FS) 三 Pl1(0) 三 Ta 二 TO e (P3.5.3) 
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(b) Compose a MAILAB program “nm3p05.m”that uses the Toutine 
“padeap ()”to generate the Pade approximation of (P3.3.1) with 7 == 
0.2 and plots it together with the Second-order Maclaurin Series expan- 
Sion and the true function (P3.5.1) for 8 = [一 9, 10]. You also run it to 
See the result as 
1 一 (7/2)9 ”一 9 十 10 


P11(09) TD8 -| 攻 二 [站 (P3.5.4) 


3.6 Rational Function Interpolation: Bulirsch-Stoer Method [S-3] 


Table P3.6 shows the Bulirsch-Stoer method, where its element in the 1 由 
Iow and the (十 1)th column is computed by the following formula: 


人 一 onHiDCR 1 本 RDDCR 1 一 Rn) 
人 一 JR 一 玉 RD 一 人 一 HiD)CR 一 民 0) 


with R2 =0andR 一 fori=1:Nandm =1:N 一 ; 
(P3.6.1) 


i+1 Pi 
玉 ， 下 及 1 


rational_interpolation(x,y,XI) 
; NI = length(xi); 


mr+1，1); RR2 = RmI) 


RR1 - R(m + 1,???); RR2 = RR2 - R(???,1 - 1); 


tmp1 = (xn-X(??3) )*RR1; 
tmp1x*(R(???,1) - R(m,?) ); 
(xn - X(?))*xRR2 -tmp1; 





Data 汪 一 测 让 三 没 1 一 3 1 一 4 


(xl 71) 有 Ri 一 沪 Ri Ri R1 
(xc2，y2) 及 一 六 R2 R2 


(xc3， 3) 及 一 妇 R3 


(xm yom) 
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(3) 


(b 


\ 必 


(C) 


The above Toutine “rational _interpolation(x,y,xi)”uses the 
Bulirsch-Stoer method to interpolate the set of data palrs (x,y) given 
as its firstsecond input arguments over a Set of intermediate points 
xi given as its third input argument. Complete the routine and apply 
it to interpolate the four data points {( 一 1 太一 D))，( 一 0.2， 太 (一 0.2))， 
(0.1, F(0.1D), (0.8,，F(0.8))} on the graph of Foxo) = 1/+8x2) for 
xi = [-100:100]/100 andplot the interpolated curve together with the 
graph of the true function 太 (x). Does it work well? How about doing 
the same job with another routine “rat_interp()”listed in Section 8.3 
of [FE-1 ?What are the values of yi([95:97]) obtained from the two 
routines? 开 you come across anything odd in the graphic results and/or 
the output numbers，what ls your explanationy7 
(cf MAILAB expresses the in-determinant 0/0 (zero-divided-by-zero) as NaN 
(Not-a-Number) and skips the value when plotting it on a graph. It may， 
therefore，be better off for the plotting purpose 让 we take no Special 
consideration into the case of in-determinant. 





Apply the Pade approximation routine “padeap()”(withM= 2 狂 N= 
2) to generate the rational function approximating Co) = 1/(L 十 8x2) 
and compare the result with the true function 矿 (x). 

To compare the rational interpolation method with the Pade approx- 
imation Scheme，apply the routines rational_interpolation() and 
padeap() (withM=3 多 AN=2)tointerpolate the four data points 
{(-2, /2), (1 AD),GFGD), CO, CO))) on the graph of 
Jox)=Simn(x) for xi = [-100:100]*pi/1100 and plot the interpolated 
curve together with the graph of the true function. How do you compare 
the approximation/interpolation results7 





Smoothness of a Cubic Spline Function 


We claim that the cubic Spline interpolation function 8(x) has the Smooth- 
ness property of 


号 CowCo)2 和 有 (PCz))? dx (P3.7.1) 


for any second-order differentiable function 太 x) matching the given grid 
points and having the Same first-order derivatives as 8y(x) at the grid points. 
This Implies that the cubic spline functions are not So rugged. Prove it by 
doing the following. 


(3) 


Check the validity of the equality 


人 Oo Co) 人 中 人 dx (P3.7.2) 


Xk 
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where the left-hand and right-hand sides of this equation are 
XK 十 1 

LHS: 1/ (zs Cdx 
3 


三 三) Cole 一 | (Cs (xz) dx 

三 万 CtDs CceHD 一 os ct 一 CCC 一 co)(P3.7.3a) 
RHS: 太 S (xz)S (xz) Cx 

二 3 Ce CH 一 SC CO 一 CGO 一 SC)  (P3.7.3b) 


(b) Check the validity of the following inequality: 
0< 1 CD 一 wGD 有 本 
1 (raDj2az -2 / DCDaz / CoCD)2ax 
(3 1 CD- 上 GoCD)2ax 


1/ (CCo)2 了 这 过 人/ 0 dx (P3.7.4) 


3.8 MAILAB Built-in Routine for Cubic Spline 
There are two MAITLAB built-in Toutines: 


>>yII 
>>yII 


Spline(x,y,XIi); 
interp1(x,y,，Xi，Sspline' ); 


Both recelive a Set of data points (x,y) and return the values of the cubic 
Spline interpolating function 8(x) for the (intermediate) points xi given as 
the third input argument. Write a program that uses these MAILAB routines 
to get the interpolation for the set of data points 


{(0, 0), (0.5, 2), (2, 一 2), (3.3, 2),， (4, 0)} 


and plots the results for [0, 4]. mn this program, append the statements that 
do the Same job by using the routine “cspline(x,y,KC)”(Section 3.3) with 
KC 一 1, 2, and 3. Which one yields the Same result as the MATLAB built- 
in routine” What kind of boundary condition does the MAILAB built-in 
Ioutine asSsume? 
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3.9 Robot Path Planning Using Cubic Spline 


Every object having a mass ls Subject to the law of inertia and So its 

Speed described by the first derivative of its displacement with respect to 

time must be continuous in any direction. In this context, the cubic spline 

having the continuous derivatives up to Second order presents a good basis 

for planning the robot path/trajectory. We will determine the path of a robot 

in Such a way that the following conditions are Satisfied: 

At ttme1 一 0 Ss, the robot starts from its home position (0, 0) with zero 

inlitial velocity,， passing through the intermediate point (1,，1) at! 上 一 1s 

and arriving at the final point (2, 4) at 上 一 2 8S. 

On arriving at (2,，4),， it starts the point at 1 一 2 S，Stopping by the 

intermediate point (3, 3) at 上 一 3 s and arriving at the point (4，2) at 

1 一 48S. 

On arriving at (4, 2), it starts the point, pasSing through the intermediate 

point (2,1) att 王 9Ss and then returning to the home position (0, 0) at 

1 一 0 8S. 

More Specifically，what we need 1S 

。the Spline interpolation matching the three points (0, 0),(1,，1),(2, 2) and 

having zero velocity at both boundary points (0, 0) and (2, 2)， 

the Spline interpolation matching the three points (2, 2),(3, 3),(4, 4) and 

having zero velocity at both boundary points (2, 2) and (4, 4), and 

the Spline interpolation matching the three points (4, 4), (3, 2), (6, 0) and 

having zero velocity at both boundary points (4, 4) and (6, 0) on the 1x 

Plane. 

On the 1y plane，we need 

。the Spline interpolation matching the three points (0, 0),(1，1),(2, 4) and 
having zero velocity at both boundary points (0, 0) and (2, 4)， 

。the Spline interpolation matching the three points (2, 4),(3, 3),(4, 2) and 
having zero velocity at both boundary points (2, 4) and (4, 2), and 

。the Spline interpolation matching the three points (4, 2),(3,，]1),(6, 0) and 
having zero velocity at both boundary points (4, 2) and (6, 0). 


Supplement the following incomplete program “robot_path”, whose objec- 
tive ls to make the required spline interpolations and plot the whole robot 
path obtained through the interpolations on the xy plane. Run it to get the 
graph as depicted in Fig. P3.9c. 


s%robot_path 
xl1=[01 2];yI= [0 1 4l];) tl= [0 1 2]; tilf= [0: 0.05: 2]; 
xit = .Cspline(ttixtlytat) ya = cspline 人 (tf yt tl)3 


"k' )，hold on 
([x1(1) x2(1) x3(1) x3(end)],[y1(1) y2(1) y3(1) y3(end)]，o ) 
([x1 x2 x3],[y1 y2 y3]，k+')，axis([0505]) 
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Figure P3.9 Robot path planning using the cubic spline interpolation. 


3.10 One-Dimensional Interpolation 


What do you have to give as the fourth input argument of the MAILAB 
built-in routine“interp1()”in order to get the Same result as that would 
be obtained by using the following one-dimensional interpolation routine 
“intrp1()”2” What letter would you see 让 you apply this routine to inter- 
polate the data points {(0,3),， (1,0),， (2,3),， (3,0),，(4,3)} for [0,4]? 


function yi = intrp1(x,y,Xi) 
M = length(x); Mi = length(xi)j; 
for mi = 1: Mi 
if xi(mi) < x(1)，yi(mi) = yY(1)-(Y(2) - 
elSseif xi(mi)>x(M) 
yi(mi) = yY(NW+(Y(N - yY(M - 1))7x(N - 
el1Se 
form= 2:M 


if xi(mi) <= Xx(m) 
yi(mi) = y(m - 1)+(Yy(m) - 
break; 
end 
end 
end 
end 





3.11 Least-Squares Curve Fitting 


(a) There are several nonlinear relations listed in Table 3.3，which 
can be linearized to fit the LS algorithm. TIhe MAILILAB routine 
“curve_fit()”implements all the Schemes that use the LS method 
to find the parameters for the template relations，but the parts for the 
relations (1]), (2), (7), (8), and (9) are missing. Supplement the missing 
parts to complete the routine. 

(b) The program “nm3p11.m” generates the 12 sets of data pairs according to 
various types of relations (人 (functions)， applies the routines 
“curve_fit() 六 sqcurvefit()” to fndthe parameters ofthe template 
relations, and plots the data pairs on the fitting curves obtained from the 
template functions with the estimated parameters. Complete and run 
to get the graphs like Fig. P3.11. Answer the following questions. 
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(人 I 玉 any, fnd the case(S) where the results of using the two routines 
make a great difference. For the case(S)，try with another initial 
guess th0 = [1 1H] of parameters, instead of th0 = [0 0]. 


(让 Ithe MAILILAB builtrin routine“1lsqcurvefit()”yields a bad 
reSult，does it always give you a warning message? How do you 
compare the two routines? 


function [th,err,yil = curve_fit(x,y,KC,C,Xxiysig) 
simplementSs the various LS curve-fitting Schemes in Table 3.5 
KC = the # of Scheme in Table 3.5 
optional constant (final value) for KC! = 0 (nonlinear LS) 
degree of approximate polynomial for KC = 0 (standard LS) 
5 Sig = the inverse of weighting factor for WLS 
Nx = length(x); X= x(:)iyY=y(:)i 
if nargin == 6，Sig = Sig(:); 
elseif Length(xi) == NX，Sig = Xi(:); xi = Xi 
else Sig = ones(Nx,1); 
end 
if nargin < 5，Xxi = Xj endj if nargin<41|1C<1，C= 1; end 
Switch KC 


case {3,4} 
A(1:Nx,:) = [x./Sig ones(Nx,1)./Sig]; 
RHS = 1og(y)./sigi th = ANRHS ; 
yi = exp(th(1)*xi + th(2)); y2 = exp(th(1)*x + th(2)); 
if KC == 3，th = exp([th(2) th(1)]); 
站 1 二 R 人 (2 XELERT 2 
end 


case 5 
if nargin < 5，C = max(y) + 1; end %final Value 
A(1:Nx,:) = [x./Ssig ones(Nx,1)./Ssig]; 


y1 =yiyi(find(y > C - 0.01)) =C - 0.01; 
RHS = 1og(C-y1)./sig; th = ANRHS ; 
yi=C- explth(1)*xi+ th(2));)y2=C- exp(th(1)*x + th(2)); 
th = [-th(1) exp(th(2))]; 
case 6 
A(1:Nx,:) = [1og(x)./sSig ones(Nx,1)./sSig]; 
y1 =yiyi(find(y < 0.01)) = 0.01; 
RHS = 1og(y1)./sigi th = ANRHS; 
yi = exp(th(1)*1og(xi) + th(2)); y2 = exp(th(1)*1og(x) + th(2)); 
th = [exp(th(2)) th(1)]; 
case 7 
Case 
Case 
otherwise s%Sstandard LS with degree C 
A(1:Nx;,C + 1) = ones(Nx,1)./Ssig; 
forn=Cc:-1:1，A(1:Nx,n) = A(1:Nxin + 1).*xj end 
RHS = y./Sigj; th = ANRHS ; 
Vy 守 th(C+1); tmp = ones(Size(xi)); 
y2 th(C+1); tmp2 = ones(Size(x)); 
forn=C:-1:1， 
tmp = tmp.*xiyj yi = yi+th(n)*tmpi 
tmp2 = tmp2.*xj y2 = yY2 + th(n)*tmp2; 
end 
end 
th = th(:)' err = norm(y - y2); 
if nargout == 0，plot(x,yY，*'，Xiiyi，，k-'); end 
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s%nm3p11 to plot Fig.P3.11 by curve fitting 
ClLear 
= [1: 20]*2 -0.1; NXx = length(x); 


noise = rand(1;NX) - 0.5; 5 1XNX random noise generator 
xi = [1:40]-0.5; %interpolation points 
figure(1)，cClf 
a=0.1;jb= -1;)c= -50) %Table 3.5(0) 
yY= arx.^2 + brx+Cc+10*noise(1:NX); 
[th,err,yil = curve_fit(x,y,0,2,xi); [a b cl，th 
[ab cl,th %if you want parameters 
f= inline('th(1)*x.^2 + th(2)*Xx+th(3)" ，th' ，Xx' ); 
[th,err] = 1Lsqcurvefit(f,[0 0 0],x,y)，yil = f(th,xIi); 
Subplot(321)，plot(x,y，*'，Xiyi，k'，xiyil nr ) 
2;jb=1iy=a./x+b+0.1*noise(1:NX); 5%5Table3.5(1) 

[th,err,yil = Curve _fit(x,y,1,0,xi); [a bl，,th 
f = inline('th(1)./X + th(2)"，th'，Xx' ); 
= [0 0]; [th,err] = lsqcurvefit(f,tho0,x,y)，yi1 = flth,Xxi); 
Subplot(322)，plot(x,y，*'，Xiyi，k'，xiyil nr ) 
= -20; b -9j Y = b./(x+a) + 0.4*noise(1:NX); %5Table 3.5(2) 
[th,err,yil = Curve _fit(x,y,2;,0,xi); [a bl,th 
f = 人 
= [0 0]; [th,err] = 1lsqcurvefit(f,th0,x,y)，yi1 = flth,Xxi); 
0 plot(Xx,yY，*'，Xiyi，k'，xiyil，r) 
.bb=0.95jyY= arb.^x+ 0.5*xnoise(1:NX); %Table 3.5(3) 
[th,err,yi]l = 0 [a bj],th 
f = inline('th(1)*th(2).^x' ，th'，Xx' )); 
th0 = [0 0]; [th,err] = lsqcurvefit(f,th0,xy)，yi1 = flth,xi); 
Subplot(324)，plot(x,y，*'，Xiyi，k'，xiyil， rr) 
a=0.1; b 1; y = bx*exp(axx) +2x*noise(1:NX); %Table 3.5(4) 
[th,err,yil = Curve _fit(x,y,4;,0,xi); [a bl,th 
f= inline(' th(2)*exp(th(1)*x)"，th' ，Xx' ); 
th0 = [0 0]; [th,err] = lsqcurvefit(f,th0,xy)，yi1 = fl(th,xi); 
Subplot(325)，plot(x,y，*'，Xiyi，k'，xiyil rr) 

= 0.1; b %Table 3.5(5) 
yY = -bxexp(-axx)jC= -min(y)+tl;jy=C+y+0.1*noise(1:Nx); 
[th,err,yil = Curve _fit(x,y,5,C,xi); [a bl，,th 
f= inline(' 1-th(2)*exp(-th(1)*x)" ，th' ，Xx' ); 
th0 = [0 0]; [th,err] = lsqcurvefit(f,th0,x,y)，yi1 = fl(th,xi); 
Subplot(326)，plot(x,y，*'，Xiyi，k'，xiyil， rr) 
figure(2)，cClf 

= 0.5jb=0.5)j yY= arx.^b +0.2*noiSse(1:NX); %5Table 3.5(6a) 
[thy,err,yi]l = Curve 下 信 ;y;0,2,Xxi); [a b],th 
f= inline('th(1)*x.^th(2)"，th' ，Xx' )); 
th0 = [0 0]; [th， 二 = 1Ssqcurvefit(f,tho0,x,y)，yi1 = fl(th,Xxi); 
Subplot(321)，plot(x,y，*'，Xiyi，k'，xiyil nr ) 

= 0.5; b -0.5; s%5Table 3.5(6b) 
yY= axrx.^b+0.05*noise(1:Nx); 
[th,err,yil = Curve _fit(x,y,6,0,xi); [a bl,th 
f= inline('th(1)*x.^th(2)"，th' ，Xx' ); 
tho = [0 0]; [th,err] = lsqcurvefit(f,th0,x,y)，yi1l = 
Subplot(322)，plot(x,y，*'，Xiyi，k'，xiyil rr) 











(ch) 开 there is no theoretical basis on which we can infer the physical relation 
between the variables, how do we determine the candidate function Suitable 
for fitting the data pairs? We can plot the graph of data pairs and choose one 
of the graphs in Fig. P3.11 which is closest to it and choose the corresponding 
template function as the candidate fitting function. 
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Figure P3.11 LS fitting curves for data pairs with various relations. 


3.12 Two-Dimensional Interpolation 


Compose a routine “z = find_depth(xiyyi)”that finds the depth z of a 
geological stratum at a point (xi,yi) given as the input arguments, based 
on the data in Problem 1.4. 


3.13 
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(cf) 下 you have no idea，insert just one Statement involving “interp2()”into 
the program “nm1p04.m” (Problem 1.4) and fit it into the format of a MAT- 
LAB function. 


Polynomial Curve Fitting by Least Squares and Persistent Excitation 


Suppose the theoretical (true) relationship between the input x and the 
output y ls known as 
yy 一 X 十 2 (P3.13.1) 


Charley measured the output data y 10 times for the Same input value 
X 一 1byusng agauge whose measurement errors has a uniform distribu- 
tion U[ 一 0.5, 十 0.5]. He made the following MAILAB program“nm3p13”， 
which uses the routine“polyfits()”to fnd a straight line fitting the data. 
(a) Check the following program and modify it 让 needed. Then, run the 
Program and see the result, Isn't it beyond your Imaginationy If you use 
the MAILAB built-in function“polyfit()”, does it get any better? 


%nm3p13 .m 
tho = [1 2]; s%true parameter 
x = ones(1,10); %the unchanged :input 


Vy tho(1)*x + tho(2)+(rand(Size(Xx)) - 0.5); 
th_ls = polyfits(x,y,1); %uses the MATLAB routine in Sec.3.8.2 
polyfit(x,y,1) %5uses MATLAB built-in function 





(b) Note that Substituting Eq. (3.8.2) into Eq.(3.8.3) yields 
QQ9 
09 | 二 [474]-477y 


=1 

加 | 达 地 六 7 | | > 1 yz 

国 M M 1M 
2 7 2 0 1 区 0 Ji 


If xz = c(constanD YV7 一 0: Mis the matrix 474 invertible? 

(c) What conclusion can you derive based on (a) and (b), with reference to 
the identifiability condition that the input must be Tich in Some Sense 
OT persistently exciting? 
(cb This problem implies that the performance of the identification/estimation 


Scheme including the curve fitting depends on the characteristic of input 
as well as the choice of algorithm . 


| (P3.13.2) 


3.14 Scaled Curve Fitting for an HIl-Conditioned Problem [M-2] 


Consider Eq. (P3.13.2),， which is a typical least-squares (LS) Solution. The 
matrix 474,， which must be inverted for the solution to be obtained， may 
become il-conditioned by the widely different orders of masgnitude of its 
elements, if the magnitudes of all zs are too large or too Small, being far 
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from 1 (see Remark 2.3). You will reallize Something about this issue after 
solving this problem. 


(3) 


(b 


S 必 


(C) 


Find a polynomial of degree 2 which fits four data points (105, 1), (1.1 x 
106,， 2)，(1.2 x 106, 9),， and (1.3 x 105，10) and plot the polynomial 
function (together with the data points) over the interval [105, 1.3 x 
106] to check whether it fits the data points well. How big is the relative 
mismatch error? Does the polynomial do the fitting job well? 


Find a polynomial of degree 2 which fits four data points (107, 1),(1.1 x 
107, 2),，(1.2 x 107, 9), and (1.3 x 107，10) and plot the polynomial 
function (together with the data points) over the interval [107, 1.3 x 
107] to check whether it fits the data points well. How big is the relative 
mismatch error? Does the polynomial do the fitting job well? Did you 
get any warning message on the MAILAB command window? What 
do you think about it? 

开 you are not Satisfied with the result obtained in (b)，why don't you 
try the Scaled curve fitting Scheme described below? 


1， Transform the xs of the data point (xn， yn)s into the region 
[一 2, 2] by the following relation. 


4 
ji 所 一 十 一 Oo 一 xzmin) (P3.14.1) 


Xmax 一 XYmin 
2. Find the LS polynomial P(xz“) fitting the data point (xz ，y)S. 
3. Substitute 


X <- 一 2 十 = 一 Xmin) (P3.14.2) 
Xmax 一 YXmin 
for x' into _P(x ). 
(cf) You can complete the following program“nm3p14”and run it to get the 
numeric anSwWers. 


s%nm3p14.m 
Clear，CLf 
format long e 
Xx= 1e6*[11.11.2 1.3]jy= [12510]; 
xi=Xx(1)+[0:1000]/1000*(x(end) - x(1)); 
[perr,yi]l = curve fit(x,y;，0,2,，Xi)j p，err 
plLlot(x,y，0' ,xiyyi)，hold on 

xmin = min(x); xmax = max(Xx); 

xf1= -2+ 4*(X-xmin)/(xmax - xmin); 

x1i = ?3??332?2?2??2??33?332?32?2?2?2?2??333?32?3232? 


了 


plot(x,y，0 ,Xiyyi) 

%5To get the coefficients of the original fitting polLynomial 
ps1 = poly2sym(p1) ; 

Syms X;j ps0 = Subs(ps1;,X，- 2 + 4/(xmax - xmin)*(x - Xxmin)); 
p0 = Sym2poly(ps0) 

format Short 
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3.1S Weighted Least-Squares Curve Fitting 


As im Example 3.7，we want to_ compare the results of applying the LS 
approach and the WLS approach for fnding a function that we can believe 
will describe the relation between the input x and the output y as 


> 一 QX ec (P3.19) 
Where the data pair (xm，ym)7S are given as 
{(1, 3.2908), (3, 3.32604), (9, 1.1640), (13, 0.3313), (17, 0.1140)} 
from gauge A with error range 士 0.1 
{(3, 4.7323), (7, 2.4149), (11, 0.3814), (13, 一 0.2396), (19, 一 0.26015)} 


from gauge B with error range 士 0.3 


Noting that this corresponds to the case of Iable 3.5(7), use the MATILAB 
Ioutine“curve_fit()”for this job and get the result as depicted in Fig. 
P3.13. Identify which one of the two lines a and b is the WL9 fitting curve， 
How do you compare the results? 























0 10 20 
Figure P3.15 The LS and WLS fitting curves to y = axebx. 


3.16 DFT (Discrete Fourier Transform) Spectrum 


Supplement the part of the MAILAB program“do_fft”(Section 3.9.2)， 
which computes the DFT spectra of the twor-tone analog Signal described by 
Eq. (3.9.2) for the cases of zero-padding and whole interval extension and 
plots them as in Figs. 3.13c and 3.13d. Which is the clearest one among 
the four Spectra depicted in Fig. 3.137 斑 you can generalize this，which 
would you choose among up-sampling，zero-padding, and whole interval 
extension to get a clear Spectrum 
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3.17 Effect of Sampling Period，Zero-Padding，and Whole Time Interval on 
DFT Spectrum 


JIn Section 3.9.2, we experienced the effect of zero-padding, sampling period 
reduction, and whole interval extension on the DFT spectrum of a two-tone 
Signal that has two distinct frequency components.， Here，we are going 
to _ investigate the effect of zero-padding，sampling period reduction，and 
whole interval extenslon on the DFT spectrum of a triangular pulse depicted 
in Fig. P3.17.1c. Additionally，we will compare the DFT with the CUET 
(continuous-time Fourier transform) and the DtFT (discrete-time Fourier 
transform) [O-1]. 

(a) The deftinition of CtFT that is used for getting the Spectrum of a 

continuous-time finite-duration Signal X(t) is 


X(w) 一 三 X(1)e-7ox di (P3.17.1) 


Oo 





1 AI A(t+2) 
convolution time 

T Shiftin 
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一 0 1 N_imesnmmg -Alt-2) 
(a) Two rectangular pulses (b) r(Dx*r(bD= 和 Ah (c) XI 人 =Alt+2) -Alt-2) 


Figure P3.17.1 Atriangular pulse as the convolution of two rectangular pulses. 


The CtUFT has several useful properties including the convolution 
Property and the time-shifting property described as 


(CUET) 
X()*ky() 一 一 > X(O)7(w) (P3.17.2) 
(CUET) 
XL 一 万 ) 一 > X(oO)e 7o0 (P3.17.3) 


Noting that the triangular pulse ls the convolution of the two rectangular 
pulse 7r( 站 7"S whose CtFETS are 


下 Sin 〇 w 
及 (w) = CtFT{r(D)} 一 1 e/21d1 一 2 一 
| CU 


we can use the convolution property (P3.17.2) to get the CtFT of the 
triangular pulse as 


7.2) 


CETIA(D} = CIETU OOD xyrOD) 全 7 RDRCw) 


sin2 mw . 2 10 
=4 一 -4sinc (三 ) (P3.17.4) 
CU 元 





PROBLEMS 173 






























2 磊 生 T T 了 2 国 生 T T 了 
， | xXa[ 咱 和 [由 
人 @ 了 鸭 1 | 1 @ 习 
用 是 四 
0 4 tit=nT 7 0 4 t=mT 7.5 
(a) xa[ml and its spectrum (b) Reduction of sampling period 
8 T T T 









一 CtFT Xe(w)| 中 4 10 
--_DtFT |Xe(9| 


一 _CtFT spectrum |Xo(w)| 
--_DtFT spectrum |Xo(9)| 
-。 DFT spectrum |Xo(k)| 


























; ， 5 ] 
人 1 | 
0 一 一 0 
0 2 4 6K 7 0 8 
0 digital frequency 工 9 2 0 digital frequency 区 2 
0 analog frequency 区 O 2 0 analog frequency 2 ww 4 
2F @ T T US 2 上 一 


| 


























-2 上 1 1 1 了 22 1 1 下 一 当 
0 8 t=mn7 15 0 8 ft=hn7 15 
(c) Zero-padding/interval extension (d) Double repetition 
8 T T T T T 下 T 
一 CtFT |X(w)| 。 澡 一 CtFT |Xo(w)| ] 
-- DiFT |IXoo | --_DtFT |Xu(9) 
-。 DFT |Xc(G)| 一 DFT |Xo(k)| 









































5 | 
尘 : 人 
也 的 0 | 
8 KK 15 0 8 Kk 15 
0 digital frequency 工 @ 2r 0 digital frequency 工 吕 2 
0 analog frequency 工 w 2 0 analog frequency 工 中 2 


Figure P3.17.2 ”Effects ofsampling period, zero-padding, and whole interval on DFT spectrum. 
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Successlively, use the time shifting property (P3.17.3) to get the CtUFT of 


xz 一 AUC+2) 一 AU 一 2) (P3.17.5) 
asS 


(3.17.3, 4) 


X(w) Toeio -Toer-j2o = 18sin(2o)sinc2 (三 ) 
元 


(P3.17.0) 
Get the CUFT 了 7(w) of the triangular wave that is generated by repeating 
X(f) two times and described as below. 


y(D) = 一 XUL 十 4) 十 YL 一 全 (P3.17.7) 


Plot the Spectrum X(w) for0 和 w<2r and check ifthe result is the 
Same as depicted in a Solid line in Fig. P3.17.2a or P3.17.2c. You can 
also plot the Spectrum X(w) for 0 和 w<4r and check if the result 
is the Same as the solid line in Fig. P3.17.2pb. Additionally，Pplot the 
Spectrum 了 (w) for 0<w<2r and check if the result is the same as 
the solid line in Fig. P3.17.2d. 

(b) The definition of DtFT，which is used for getting the Spectrum of a 
discrete-time Signal x[m]， is 


X(Q) = 六 X[m]e 一 Sn (P3.17.8) 


1 一 一 CO 


Use this formula to compute the DUFTS of the discrete-time Signals 
Xa[2], xp[2], xc[2], xz[2] and plot them to see 让 the results are the 
Same as the dotted lines in Fig. P3.17.2a-d.， What is the valid ana- 
log frequency range over which each DFT Spectrum is Simjlar to the 
Corresponding CtUFT Spectrum, respectively? Note that the valid analog 
frequency Tange is [一 交 / 了 ,十 T/T] for the Sampling period 了 . 

(c) Use the definition (3.9.1a) of DFT to get the Spectra of the discrete-time 
Signals xz[2], xpo[2], xc[2], and xz[2] and plot them to see 寺 the results 
are the Same as the dots in Fig. P3.17.2a-d. Do they match the Samples 
of the corresponding DUFTS at 人 一 2K7r/AN7 Among the DFT spectra 
(al)，(b)，(c),， and (d)，which one describes the corresponding CUFT or 
DFT Spectra for the widest range of analog frequency? 


3.18 Windowing TIechniques Against the Leakage of DFT Spectrum 


There are Several window functions ready to be used for alleviating the 
Spectrum leakage problem or for other purposes. We have made a MAT- 
LAB routine“windowing()”for easy application of the various windows. 
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Applying the Hamming window function to the discrete-time Signal xz[7] 
in Fig. 3.13d，get the new DFT spectrum, plot its magnitude together with 
the windowed Signal, check if they are the Same as depicted in Fig. P3.18b， 
and compare it with the old DFT spectrum in Fig. 3.13d or Fig. P3.18a. 
You can start with the incomplete MAILAB program“nm3p18.m”below. 
What ls the effect of windowing on the Spectrum 
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Figure P3.18 The effect of windowing on DFT spectrum. 


function xw = Windowing(x,W) 

N = Jength(x) ; 

if nargin <21WwW== rt' | Isempty(w)，xw = X; 
elseif w== 'bt' ，xw= Xx.*bartlett(N) 
elseif WwW == 'bk'，xw= X.*blackman(N) ; 
elseif w== 'hm'，xw= Xx.*hamming(N) ; 

end 


%nm3p18: windowing effect on DFT Spectrum 
w1 = 1.5*pi; WwW2 = 3*pi;i %two tones 
N=64jn= 1:N;T=0.1;)t= (nn - 1)*T; 
k= 1:Ni WwW0O = 2*pi/TJw= (k - 1)*woi 

xbn = Sin(w1x*t) + 0.5*Sin(W2*t) ; 

xbwn = Windowing(xbn，bt ) ; 

xb = fft(xbn); Xbw = fft(xbwn) ; 
Subp1lot(421)，Sstem(t,xbn， . ) 
Subplot(423)，stem(k,abs(Xxb)，,，) 
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3.19 Interpolation by Using DFS: Zero-Padding on the Frequency Domain 
The fitting curve in Fig. 3.14d has been obtained by zeroing out all the 
digital fredquency components higher than T/2 [rad](W4 < 上 < 3N/4) of the 
sequence Y[1] in Fig. 3.14a. Plot another fitting curve obtained by removing 
all the frequency components higher than /4 [radj(NMW8 < 上 < 7N/8) and 
compare it with Fig. 3.14d. 


3.20 On-Line Recursive Computation of DFT 


For the case where you need to compnute the DFT of a block of data every 
time a new Sampled data replaces the oldest one in the block，we derive 
the following recursive algorithm for DFT computation. 

Defining the first data block and the th data block as 


{xo[0], xzo[,...，,xo[LNV 一 J)}={10,0,.. .0} (P3.20.1) 
{w[0] ,xD ,xnwLN 一 J)={xz[2x2 十 ] ,xm 二 AN 一 1 (GOP3.20.2) 


the DFT for the (7 十 1H)th data block 


{zm+1[0], zs XoHIIN 一 十 = 全 十 xD 十 熙 xD 十 和 


(P3.20.3) 
can be expressed in terms of the DFT for the th data block 
N=-1 
和 on 一 》 xn[m]eProUN 大 一 0:N 一 1 (P3.20.4) 
1 一 0 


as follows: 
N-1 于 
XI 一 六 xDoe-7PrnkN 二 村 志 思 十 1]e-72rakN 
一 区 十 1]e-7Pzro+DMNej2rkN 
mn-0 


TV ， 
= 了 》 本 
天 王 


呈 1 XI]e-PrnkN 十 x[N] 一 xl| ej2mrK/N 
一 {X( 十 YLN] 一 x[0]jez24 (P3.20.5) 


You can compute the 128-point DFT for a block composed of 128 random 
numbers by using this RDFT algorithm and compare it with that obtained 


PROBLEMS 177 


by using the MAILAB built-in routine“fft()”. You can Start with the 
incomplete MAILAB program“do_RDFT.m”below. 


%do_RDFT 

Clear，CLf 

N= 128;)Kk= [O:N - 1]) 

X zeros(1,N); %initialize the data block 
Xr = zeros(1;N); 5 and its DFT 

form= 0O:N 


XxN = rand;i %new data 
XF = 
Xx= [x(2:N) XN]; 
end 
dif = norm(Xr-fft(x)) %5difference between RDFT and FFT 





NONLINEAR EQUATIONS 





4.1 ITERATIVE METHOD TOWARD FIXED POINT 


Let's see the following theorem. 


Fixed-Point Theorem: Comtractiom 77jeoremz 攻 2 Secton 51。Suppose a function 
gt) is defined and its first derivative 8 (x) exists continuously on some interval 
Z 一 [2 一 六 X 十 门 around the fxed point x“ of g(xz) such that 


8 ) 一 并 7 (4.1.1) 


Then, if the absolute value of g'(xz) is less than or equal to a positive number w 
that ls strictly less than one, that ls， 


lg COJI<w <1 (4.1.2) 
the iteration Starting from any point xo E 了 
XkH1l 一 8 With xzo E 7 (4.1.3) 


conversges to the (unique) fxed point x2 of g(x)， 


4PPlied Nu1aerical Metpods Using MA4774B9， by Yang, Cao, Chung, and Morris 
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Proof The Mean Value Theorem (MVT) (Appendix A) Says that for any two 
points xo and x?", there exists a point X between the two points Such that 


4.1.3),(4.1.1 
oo (1320410 


SG0) 一 SG ) 一 8 CGO 一 XXX 8 Co)GCo 一 xz2) 0) 


Taking the absolute value of both sides of (1) and using the precondition 
(4.1.2) yields 
一 X Ecolxzo 一 2 < xzo 一 2 (2) 


which implies that xl ls closer to x? than xo and thus still stays inside the interval 
7. Applying this succesSlively， we can get 


| 和 不 一 xz < ax 一 xl<o2lxea 一 xl< <owkxo 一 xl 一 0asK 一 co 0) 
which implies that the iterative Sedquence {Xk} generated by (4.1.3) converges to X2. 


(Q) Is there any possibility that the fxed point ls not unique 一 that js, more than one 
point satisfy Eq. (4.1.1) and so the iterative Scheme may get confused among the 
Several fxed points? 

(A) It can never happen, because the points x21 and x2” satisfying Eq. (4.1.1) must 
be the same: 


X21 一 2X22| 一 |g(Gx20) 人 SO)| < ax21 一 X22| (a 二 J); |x21 一 各 ?| 二 0; X21 二 22 


In order to solve a nonlinear equation jx) 一 0 using the iterative method based on 
this fxed-point theorem，we must Somehow arrange the equation into the form 


X 一 8&Cx) (4.1.4) 


and start the iteration (4.1.3) with an initial value xo，then continue until Some stop- 
ping criterion ls satisfied; for example, the difference |xk+l 一 2 between the SucceSssive 
iteration values becomes Smaller than Some predefined number (ToLX) or the iteration 
number exceeds Some predetermined number (MaxIter). This Scheme is cast into the 
MAILLAB routine“fixpt()”. Note that the second output argument (err) is never the 
real error 一 that ljs, the distance to the true Solution 一 but just the last value of |xzk+l 一 3 
as an error estimate. See the following remark and examples. 


了 R 了 emark 4.1. Fixed-Point Iteration. Noting that Eq. (4.1.4) is not unique for a 
given jx) = 0, it would be good to have 8g(C) such that |8(xz)| < 1 inside 
the interval 7 containing lts fxed point xz” which is the solution we are look- 
ing for. It may not be So easy，however, to determine whether |8g'Cc)| < 1 is 
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satisfied around the solution point 寺 we don't have any rough estimate of the 
Solution , 


function [xerr,xx] = fixpt(g,Xx0,TOoLX,MaxIter) 
Solve X = g(x) starting from Xx0O by fixed-point iteration . 
Input : g,x0 = the function and the initial guess 
TolX = upperbound of incremental difference |xn + 1) - xn)| 
MaXxIter maximum ## of iterations 
Output: X point which the algorithm has reached 
err last value |x(k) - x(k - 1)| achieved 
XX history of X 
if nargin < 4，MaxIter = 100; end 
if nargin < 3，TolX = 1e-6;j end 


Xx(1) = X0; 
for k = 2:MaxIter 
Xx(k) = feval(g,xx(k - 1)); ssEq.(4.1.3) 
err = abs(xx(k) - xx(kK - 1))j if err < TolX，break;i end 


= XX(k); 
If k == MaxIter 

fprintf('Do not rely on me，though best in %q iterations\n ,MaxIter) 
end 





上 Example 4.1，Fixed-Point Iteration. Consider the problem of solving the nonlin- 
ear equation 


Jo)= 所 一 2=0 (E4.1.1) 


In order to apply the fxed-point iteration for solving this equation，we need 
to convert it into a form like (4.1.4). Let's try with the following three forms and 
guess that the solution ls in the interval 7 = (1, 1.5). 





(a) How about xz 一 2=0 一 妇 一 2 一 X 一 2 一 gu)? (E4.1.2) 


Let's See if the absolute value of the first derivative of guo(x) ls less than 
one for the solution interval，that is，|8ge (xz)| 一 2]x2 < 1VYxXE7. This 
condition does not Seem to be satisfied and So we must be pessimilstic 
about the possibility of reaching the solution with (上 E4.1.2). We don't need 
many iterations to confirm this. 
X0 一 ]; 二 人 二 二 芝 二 六 0 
0 1 2 
(E4.1.3) 
The iteration turned out to be Swaying between 1 and 2, never approaching 
the Solution. 
(b) How about z2 一 2=0 一 G 一 D?+2xz 一 3 一 0 一 工 = 一 计 fc 一 D? 
3} 一 So(x)7 (上 E4.1.4) 


This form seems to Satisfy the convergence condition 





lg Cl = 一 II<053<1 VXET (E4.1.5) 
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(C) 
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and So we may be optimistic about the possibility of reaching the solution 
with (上 E4.1.4). To confirm this，we need just a few iterations，which can 
be performed by using the routine“fixpt()”. 


>>gb=inline(' -((Xx-1).^2-3)/2 ，Xx' ); 

>>[Xx,err,Xx]=fixpt(gb,1,1e-4;,50) 

>>XX 
1.0000 1.5000 1.3750 


1.4297 1.4077 


The iteration is obviously convergingtothetrue solution V2 = 1.414...， 
which we already know in this case. This process is depicted in Fig. 4.1a. 


How about xx2 一 2 一 一 一 x+ 一 二 十 X 一 工 一 寺 (x 十 二 ) 一 
SecCOD)7 (E4.1.6) 
This form seems to Satisfy the convergence condition 
1 2 
lg (xz)| 三 二 1 一 一 | 和 0.3 一 1 YX ET (E4.1.7) 
2 2 





which guarantees that the iteration will reach the solution. Moreover, Since 
this derivative becomes zero at the solution of xz = 2, we may expect fast 
Convergence，which is confirmed by using the routine“fixpt()”. The 
process ls depicted in Fig. 4.1b. 


>>gc = inline(' (x+2./X)/12 ，X' ); 

>>[Xxierr,Xxx] = fixpt(gc,1,1e-4;,50) 

>>XX 
1.0000 “1.5000 1.4167 


1.4142 1.4142 


(cf) In fact 过 the nonlinear equation that we must Solve is a polynomial equation， 
then it is convenient to use the MAILAB built-in command“roots ()”. 









































1.55 | | - 1.55 Fr | - 
儿 =X/ SS 儿 =X7 
1.5 上 --- 人 -1 1.5| 一 -1 
YY= go 人 1 YY=goo 2 
1.45 上 上 ， 2 1.45 上 SR 7 
1， 10000 ee ， 1.0000、、、 人 
个 1.5000 人 二 个 1.5000 -fc 
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Figure 4.1 Iterative method to solve nonlinear equations based on the fixed-point theorem. 
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(Q) How do we make the iteration conversge to another Solution X 一 一 V2 of 
2 
太一 2 一 0) 


4.2 BISECTION METHOD 


The bisection method can be applied for solving nonlinear equations like jx) 一 
0, only in the case where we know Some interval [a, pg] on which 太 (x) is contin- 
uous and the solution uniquely exists and, most Importantly， 太 (ac) and 太 () have 
the opposite Signs. The procedure toward the solution of jx) = 0 is described 
as follows and is cast into the MAITLAB routine“bisct()”. 


Step 0. Initialize the iteration number 一 0. 

Step 1 Let7a 一 3(a 十 四 .于 太 az) 久 0orz(B 一 0) 久 0,then stop the iteration ， 

Step 2. If 太 C) j2) > 0, then let a 擂 11; otherwise, letp < 710. Go backto 
Step 1. 


function [xerr,xx] = bisct(f,ab,TolLX,MaxIter) 
%bisct.m to Solve f(x) = 0 by using the bisection method . 
s%input : 咎 = ftn to be given as a String 'f' if defined in an M-file 
a/b = initial Jeft/right point of the Solution interval 
TOJX Upperbound of error |x(k) - xo| 
MaXxIter maximum # of iterations 
Output: X = point which the algorithm has reached 
err (bp - a)/2(half the last interval Width) 
XXx = history of X 
TolFun=epsi fa = feval(f,a); fb = feval(f,b); 
if faxfb > 0，error(' We must have f(a)f(b)<0! ); end 
for kK : MaxIter 
xx(k) = (a + b)/2; 
fx = feval(f,xx(k));j err = (b-a)/2; 
if abs(fx) < TolFun | abs(err)<TolX，break; 
elseif fxx*fa > 0，a = XX(k); fa = fxi 
else b = xx(k); 
end 


= xx(k) ; 
if k == MaxIter，fprintf('The best in %5d iterationsNn ,MaxIter)，end 





了 emark 4.2. Bisection Method Versus Fixed-Point Iteration 


1. Only if the solution exists on Some interval [wa, p], the distance from the 
midpoint (a 十)/2 of the interval as an approximate Solution to the true 
solution ls at most one-half of the interval width 一 that is, ( 一 C)/2, which 
wetake as a measure of error. Therefore, for every iteration of the bisection 
method, the upper bound of the error in the approximate Solution decreases 
by half. 
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2. The bisection method and the false position method appearing in the next 
Section will deftinitely give us the solution，only 革 the solution exists 
uniquely in Some known interval. But the convergence of the fxed- 
point iteration depends on the derivative of g(x) as well as the initial 
value X0. 


3. The MATILAB built-in routine fzero(f,x) finds a zero of the function 
given as the first input argument, based on the interpolation and the bisec- 
tion method with the initial solution interval vector x 一 [a b] given as 
the Second input argument. The routine ls Supposed to work even with an 
initial guess x 一 Xo of the (Scalarm) Solution，but it Sometimes gives us a 
WwWIiong result as illustrated in the following example. Therefore, it is safe 
to use the routine fzero() with the initial solution interval vector [a D] as 
the second input argument. 


了 kxXample 4.2. Bisection Method. Consider the problem of solving the nonlinear 
equation 
Ja2(x) 王 tan(r 一 ZX) 一 XY 一 0 (E4.2.1) 


Noting that .Ai2z(x) has the value of infinity at x 一 T/2 一 1.37...，We Set 
the initial solution interval to [1.6, 3] excluding the Singular point and use the 
MAILAB routine“bisct()”as follows. The iteration Seems to be converging 
to the solution as we expect (See Fig. 4.2b). 


>>f42 = inline('tan(pi - X)-X' ，X'); 
>>[xerr,Xx] = bisct(f42,1.6,3,1e-4,50) |; 
>>XX 
2.3000 1.9500 2.1250 2.0375 1.9937 2.0156 ..， 2.0287 


But if we start with the initial solution interval [a, p] such that 太 c) and 三 (D) 
have the Same Sign，we will face the error message. 


>>[xerr,Xx] = bisct(f42,1.5,3,1e-4,50) |; 
??? Error Using ==> bisct 
We must have f(a)f(b)<01! 


Now, lefs see how the MAILAB built-in routine fzero(f,Xx) Works. 


>> fzero(f42,[1.6 3]) 
ans = 2.0287 5%5good jobl 


>> fzero(f42,[1.5 3]) 
??? Error using ==> fzero 
The function values at interval endpoints must differ in Sign， 


>> fzero(f42,1.8) %with an initial guess as 2nd input argument 
ans = 1.5708  %wrong result with no warning message 


(cf) Not all the solutions given by computers are good, especially when we are carelesS. 
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K| ak Xk br 帮 X0 

0 | 1.6 3.0 32.6, -2.86 

1|116 |12.3 3.0 一 1.1808 

2| 1.6 |11.95 2.3 0.5595 

3 | 1.95 |2.125 2.3 -0.5092 

4 | 1.95 12.0375 | 2.125 -0.5027 

1.6 1.8 2 22 24 26 2.8 
(a) Process of the bisection method (b) The graph of fg =tan( 一 为 一 X 


Figure 4.2 Bisection method for Example 4.2. 


4.3 FALSE POSITION OR REGULA FALSIMETHOD 


Similarly to the bisection method, the false position or regula falsi method starts 
with the initial solution interval [ac, p] that ls believed to contain the solution of 
J(x) = 0. Approximating the curve of Fx) on [a,p]by astraight line connecting 
the two points (a,， 太 (Ga)) and (，F())， it guesses that the solution may be the 
point at which the straight line crosses the X axXis: 


J/ (0) J/ (DO) 


=a- 一 -四 = 这 
ja) 一 太 D) 


7 - Fo) ”7 一 OO) 
(4.3.1) 


function [xerr,xx]l = falsp(f,ab,TolLX,MaxIter) 
%bilisct.m to Solve f(x)=0 by using the false position method . 
s%input : 咎 ftn to be given as a String 'f' if defined in an M-file 
ay/b initial left/right point of the Solution interval 
TOJX Upperbound of error(max(|x(k)-al,|b-x(k)|1)) 
MaXxIter maximum # of iterations 
Output: X = point which the algorithm has reached 
err = max(x(]Jast)-al,|b-x(1Last)|) 
XX history of X 
TolFun = epsj fa = feval(f,a); fb=feval(f,b); 
if faxfb > 0，error(' We must have f(a)f(b)<0! ); end 
for k = 1: MaxIter 
Xx(k) = (axfb-bx*fa)/(fb-fa);j %5Eq.(4.3.1) 
fx = feval(f,xx(k)); 
err = max(abs(xx(k) - a)，abs(b - xx(k))); 
if abs(fx) < TolFun | err<TolX，break 
elSelif fxr*fa > 0，a = XX(k); fa = fxj 
elsSe b= xx(k); fb = fx; 
end 


XXx(Kk) ; 
if k == MaxIter，Tfprintf( The best in %dq iterationsN\n ,MaxIter)，end 
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(a) Bisection method (b) False position method 


Figure 4.3 Solving the nonlinear equation fg =tan(r 一 X 一 X=0. 


FEor this method, we take the larger of lx 一 dl| andle 一 xl as the measure of error. 
This procedure to Search for the Solution of jx) = 0 is cast into the MAILAB 
routine“falsp ()”. 

Note that although the false position method aims to Improve the convergence 
Speed over the bisection method, it cannot always achieve the goal，especially 
when the curve of 让 (xz) on [a,p] is not well approximated py a straight line as 
depicted in Fig. 4.3. Figure 4.3b shows how the false position method approaches 
the Solution, started by typing the following MAILAB statements, while Fig. 4.3a 
Showsgs the footprints of the bisection method. 


>>[Xxverr,Xxx]l = falsp(f42,1.7,3,1e-4,50) %with Initial interval [1.7,3] 


4.4 NEWTON(-RAPHSON) METHOD 


Consider the problem of fnding numerically one of the Solutions，x2,， for a 
nonlinear equation 
Ja)=C 一 zx 0) 8Co0)=0 


where jx) has (xx 一 XY2) (1 ls an even number) as a factor and So its curve 
is tangential to the x-axis without crossing it at x 一 X2. In this case, the Signs 
of 太 ix 一 5) and jx2? 十 s) are the Same and we cannot fnd any interval [a, bo] 
containing only x?” as a Solution Such that (ao) J(B) < 0. Consequently，brack- 
eting methods Such as the bisection or false position ones are not applicable to 
this problem. Neither can the MAILAB built-in routine fzero() be applied to 
solve as simple an equation as x?2 = 0, which you would not believe until you try 
it for yourself. Then, how do we Solve it? The Newton(-Raphson) method can 
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be used for this kind of problem as well as general nonlinear equation problems， 
only if the first derivative of 太 (x) exists and ls continuous around the solution. 

The strategy behind the Newton(-Raphson) method ls to approximate the 
curve of jx) by its tangential line at Some estimate XK 


> 一 Cab = 广 CeGCc 一 z) (4.4.1) 


and set the zero (crossing the XY-axis) of the tangent line to the next estimate 
大 十 1 ， 
0 一 Je = 三 CC 一 也) 
J Co) 


4.4.2 
矿 (Cxp) ) 





XK+1 一 XK 一 


This Newton lterative formula is cast into the MAILAB routine“newton()”， 
which is designed to generate the numerical derivative (Chapter 9) in the case 
where the derivative function ls not given as the Second input argument， 

Here, for the error analysis of the Newton method，we consider the second- 
degree Taylor polynomial (Appendix A) of Frx) about x 一 Xk: 


二 夺 c 本 


function [xfx,Xxx]l = newton(f,df,x0,ToLX,MaxIter) 
%newton.m to Solve f(x) = 0 by using Newton method . 
input: f= ftn to be given as astring 'f' if defined in an M-file 
df = df(x)/dx (If not given，numerical derivative 1Ss Used.) 
x0 = the initial guess of the Solution 
TolX = the upper 1lLimit of |x(k) - xX(k-1)| 
MaxIter the maximum # of iteration 
Output: X the point which the algorithm has reached 
fx = Tf(X(LJast))，Xxx = the history of X 
1e-4; h2 = 2*h;j ToLFun=eps; 
if nargin == 4 & isnumeric(df)，MaxIter = ToOlLlX; TolX = xX0;j x0 = dfj end 
Xx(1) = xX0j fx = feval(f,x0); 
for k = 1: MaxIter 
If ~ISsnumeric(df)，dfdx = feval(df,xx(k)); %5derivative function 
else dfdx = (feval(f,xx(k) + h)-feval(f,xx(k) - h))/Vh2; %numerical drv 
end 
dx = -fx/dfdx; 
XXx(k+1) = XXx(k)+dx;i %5Eq.(4.4.2) 
fx = feval(f,xx(k + 1)); 
if abs(fx)<TolFun | abs(dx) < TolX，breakj end 
end 


XXx(k + 1); 
if k == MaxIter，fprintf(' The best in %5d iterationsNn ,MaxIter)，end 
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We Substitute XY 一 X2 (the Solution) into this and use 太 (x2) = 一 0 to write 





2 
一 Xp) 


0= jc s oo 十 万 CoDGe 一 区 ) 十 / 2 or 


and 


二 十 让 时 2 三 


Substituting this into Eq. (4.4.2) and defining the error of the estimate XK as 
EKE 一 其 一 X2 Wecan get 


这 O Ji) O 沁 
MEHL 祥 克 十 (人 一 大 ) 十 2 (CC” 一 其 ) 
- ) 
|ek+1| 饼 二 间 ， = 4ket = |4kekllek| (4.4.3) 








This implies that once the masgnitude of initial estimation error |eol ls Small 
enough to make |4eo| < 1 the masgnitudes of Successive estimation errors get 
Smaller very quickly so long as 4 does not become large. The Newton method 
is Said to be “quadratically convergent” on account of the fact that the magnitude 
of the estimation error ls proportional to the square of the previous estimation 
elTOT. 

Now', it is time to practice using the MAILAB routine “newton()”for solving 
a nonlinear equation like that dealt with in Example 4.2. We have to type the 
following statements into the MAILAB command window. 


>>x0 = 1.8; TolX = 1e-5; MaxIter = 50;j %wWith Initial guess 1.8,... 
>>[xierr,xx] = newton(f42,Xx0,1e-5;50) %1st order derivative 
>>df42 = inline('-(sec(pi-x)).^2-1'，'Xx'); 5%51st order derivative 
>>[xerr,Xxx1] = newton(f42,df42,1.8,1e-5,50) 


了 R 了 emark 4.3. Newton(-Raphson) Method 


1， While bracketing methods such as the blisection method and the false posi- 
tion method converge in all cases, the Newton method ls guaranteed to 
converge only in case where the initial value xo is sufficiently close to the 
Solution x2 and 4() 三 | JJ) 人 2 is sufficiently small for x 之 7. 
Apparently, it is good for fast convergence if we have Small 4(xz) 一 that ls， 
the relative magnitude of the second-order derivative | jz)l over | 疡 Co)lis 
small. In other words, the convergence of the Newton method is endangered 
if the Slope of Foxc) is too fat or fuctuates too Sharply， 

2. Note two drawbacks of the Newton(-Raphson) method. One ls the effort 
and time required to compute the derivative (xb) at each iteration; the 
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Figure 4.4 Solving nonlinear equations (0 = 0 by using the Newton method. 


other ls the possibility of going astray, especially when 太 (x) has an abruptly 
changing Slope around the solution (e.g., Fig. 4.4c or 4.4d), whereas it con- 
verges to the solution quickly when 太 (x) has a steady Slope as illustrated 
in Figs. 4.4a and 4.4b. 


4.5 SECANT METHOD 


The secant method can be regarded as a modification of the Newton method in 
the sense that the derivative ls replaced by a difference approximation based on 
the SuccesSive estimates 


Jp) 一 Je) 


MXK 一 KK 一 1 


PC (4.5.) 


which is expected to take less time than computing the analytical or numerical 
derivative. By this approximation, the lterative formula (4.4.2) becomes 


Fo 7Gb - GD) 
0 vim re (4.5.2) 





KE 二 1 一 XK 
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function [xfx,Xx] = Secant(f,x0,TolIX,MaxIter,Vvarargin) 
Solve f(x) = 0 by using the secant method , 
input : 千 = ftn to be given as a String 'f' if defined in an M-file 
x0 = the initial guess of the Solution 
TolX = the upper Limit of |x(k) - x(k - 1)| 
MaxIter = the maximum # of iteration 
Output: X = the point which the algorithm has reached 
TfX f(x(Llast))，Xx = the history of X 
1e-4j h2 = 2*hj ToLFun=eps 
Xx(1) = x0; fx = feval(f,xo,varargin{f:}); 
for k = 1: MaxIter 
ifk<= 1，dfdx= (feval(f,xx(k) + hvarargin{f:})-.,. 
feval(f,xx(k) - hvarargin{t:}))7h2; 
else dfdx = (fx - fx0)/dxi 
end 
dx = -fx/dfdx; 
xx(kK+ 1) = XX(k) + dxji 5%5Eq.(4.5.2) 
fx0O = fx; 
fx = feval(f,xx(k+1)); 
if abs(fx) < TolFun | abs(dx) < TolX， break;j end 


= XX(k + 1); 
if k == MaxIter，Tfprintf( ' The best in #%q iterationsNn ,MaxIter)，end 





This secant iterative formula is cast into the MAITLAB routine“secant () ”， 
which never needs anything like the derivative as an input argument，We can 
use this routine“secant()”to solve a nonlinear equation like that dealt with 
in Example 4.2, by typing the following statement into the MAILAB command 
window. The process is depicted in Fig. 4.5. 


>>[x,err,Xxx] = Secant(f42,2.5,1e-5,50) %wWwith Initial guess 1.8 
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Figure 4.5 Solving a nonlinear equation by the secant method. 
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4.6 NEWTON METHOD FOR A SYSTEM OF NONLINEAR EQUATIONS 


Note that the methods and the corresponding MAILAB routines mentioned so 
far can handle only one scalar equation with respect to one Scalar variable. In 
order to see how a System of equations can be Solved numerically，we rewrite 
the two equations 


万 (xl, xz2) 三 0 


4.0.1 
户 (xl xz2) 三 0 


by taking the TIaylor series expansion up to first-order about some estimate point 
(XIE， 2X2K) as 























六 9j 37 
万 (xl 2) 宕 万 (KCl X2k) 十 (xl 一 XI6) 十 了 Gocz 一 X2t) 一 0 
XI1 Gaksaon X2 Coaksxon 
0 0 
户 (xl, 2) 三 户 (xzlk, 2k) 十 玖 Cl 一 XI 十 2 (xz 一 X2t) 一 0 
妈 CCIK,X2K) 2 (CCIE,X21) 
(4.6.2) 
This can be arranged into a matrix-Vvector form as 
万 GD2) | 全 | 万 Ceney xb 十 0 证 /axzl 97/0x2 X1 一 XIK 
(cl 72) 万 (CC 21) 0j2/0xz1 jp/9z2 | oz 一 2 
到 国 (4.6.3) 
Which we Solve for (xl, x2) to get the updated vector estimate 
一 1 
XI | | 2 | aar 3977/9xz 万 (Cxlk, 7X21) 
2X2.E+H1 2 0j2710xz1 90727922 |] | 上 户 Cak xb 
(4.6.4) 


xl 三 欢 一 JIfx with the Jacobian 及 Cam) [9 记 /8xzn]lw 


This is not much different from the Newton iterative formula (4.4.2) and is cast 
into the MATILAB routine“newtons ()”. See Eq. (C.9) in Appendix C for the 
definition of the Jacobian. 

Now, let's use this routine to solve the following System of nonlinear equations 


X1 十 4x2 一 5 
(4.6.5) 
2x1 一 2xl 一 3xa = 2.5 


In order to do so，we Should first rewrite these equations into a form 1like 
了 Eq. (4.60.1) as 
ocxz) 一 x+40 一 5=0 


(4.6.6) 
户 C xz)=2x 一 2x 一 3z 一 25=0 
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function [xfx,Xx] = newtons (f,Xx0,TolLX,MaxIter,Vvarargin) 
%newtons.m to Solve a Set of nonlinear eqs f1(x)=0，f2(Xx)=0，. ， 
不 1^st-order vector ftn equivalent to a Set of equations 
X0 = the Initial guess of the Solution 
TOJX the Upper Limit of |xX(k) - x(k - 1) 
MaxIter the maximum # of iteration 
= the point which the algorithm has reached 
f(x(1Last)) 
the history of X 
1e-4; TolFun = eps;j EPS = 1e-6; 
feval(f,xo,varargin{:})i 
= Jength(fx); NX = Length(x0); 
Nf ~= NX，error(' Incompatib1le dimensions of f and x0! ); end 
nargin < 4，MaxIter = 100; end 
nargin < 3，TolXx = EPS; end 
Xx(1,:) = xX0(:).'; %5Initialize the Solution as the Initial row Vector 
%5fxoO = norm(fx) ; s%5(1) 
for k = 1: MaxIter 
dx = -jacob(f,xx(k,:),h,varargin{f:})NXfx(:); /35%-[dfdx] -1*fX 
%5for J] = 1:; 3 %damping to avoid divergence 55(2) 
%5dx = dxXx/2; 5(3) 
Xx(kK+ 1,，:) = Xx(k,:) + dx ; 
fx = feval(f,xx(k + 1,:)，,varargin{f:}); fxn norm(fx) ; 
% if fxn < fx0o， breakj end (4) 
s%5end (5) 
if fxn < TolFun | norm(dx) < TolX，breakj end 
%5fx0 = fxnj; s%5(6) 
end 
= XX(kK + 1)，:); 
if k == MaxIter，Tfprintf( The best in %#%q iterationsNn ,MaxIter)，end 


function g = jacob(f,xih,varargin) %Jacobian of f(Xx) 
if nargin < 3，h= 1e-4; end 
h2 = 2x*hj N= length(x);}X= x(:).)I= eye(N); 
forn= 1:N 
gl(:,n) = (feval(f,x + I(n,:)x*h,varargin{:}) ... 
-feval(f,Xx - I(n,:)*h,varargin{ft:})) /hn2i 
end 





and convert it into a MAILAB function defined in an Mr-file，say,“f46.m” 
as follows. 


function y = f46(X) 
X(1)*x(1) + 4*X(2)*Xx(2) 


2xX(1)*xXx(1)-2x*Xx(1)-3xX( 





Then，we type the following statements into the MAILAB command window: 


>>X0O 
X 


[0.8 0.2]; x = newtons( 'f46' ,X0) %initial guess [.8 .2] 
2.0000 0.5000 
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Figure 4.6 ”Solving the set (4.6.6) of nonlinear equations by vector Newton method. 


Figure 4.6 Shows how the vector Newton iteration may proceed depending on 
the initial guess (xlo, X20). With (xlo, X20) 一 (0.8, 0.2)，it converges to (2, 0.3)， 
which is one of the two roots (Fig. 4.6a) and with (xlo, x20) 一 (一 1 0.9), it con- 
verges to (一 1.2063, 0.9413)，which is another root (Fig. 4.0b). However，with 
(xlo, X20) 一 (0.9, 0.2),it wanders around as depicted in Fig. 4.6c. From this fgure， 
we can See that the iteration ls jumping too far in the beginning and then going 
astray around the place where the curves of the two functions 万 (x) and 户 (x) 
are close, but not crossing. One idea for alleviating this problem is to modify the 
Newton algorithm in such a way that the step Size can be adjusted (decreased) to 
keep the norm of fxx) from increasing at each iteration. The so-called damped 
Newton method based on this idea will be Implemented in the MAILAB routine 
“newtons ()” 寺 you activate the Six Statements numbered from 1 to 6 by deleting 
the comment mark(s) from the beginning of each line. With the same inltial guess 
(xlo, X20) 一 (0.9, 0.2) as in Fig. 4.6c, the damped Newton method successfully 
leads to the point (2, 0.3), which is one of the two roots (Fig. 4.6d). 

MAILAB has the built-in function“fsolve(f,x0)”，which can give us a 
Solution for a System of nonlinear equations. Let us try it for Eq. (4.6.5) or (4.0.0)， 
which was already defined in the M-fhile named “f46.m . 


>>X fsolve( 'f46' ,X0,optimset( ' fsolve ' )) %with default parameters 


Xx = 2.0000 0.5000 


4.7 SYMBOLIC SOLUTION FOR EQUATIONS 


MAILAB has many commands and functions that can be very helpful in dealing 
with complex analytic (Symbollic) expressions and equations as well as in getting 
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numerical solutions. One ofthem is“solve()”, which can beused for obtaining the 
Symbolic or numeric roots of equations. According to what we could see by typing 
"help solve'” into the MAILAB command window, its usages are as follows: 


>>SOlvel('px*Sin(x) = ) s%regarding x as an unknown variable and p as a parameter 
ans = asin(r/p) ssin-10/p) 
2 DLLVBT  X 人 2 二 于 人 2 


xl1 = [ 2.1] x2 = [ 0.500000] 
[ -1.206459] [ 0.941336] 
[0.603229 -0.392630x [-1.095668 -0.540415e-1xi] 


] 
[0.603229 +0.392630*i] [-1.095668 +0.540415e-1*il] 
>>S = SoOlve('x*3 -y?3=2'，x=y') s%returns the Solution in a structure . 
S = x: [3xlf syml] 
y: [3x1 sym] 


ans = 1] 
上 二 Ht2<LASeTT72 
[ -1/2-1/2*i*3^(1/2)] 





ans = -1] 
[二 2 2 
[2 

>>[UV] = Solvel('aru'2 +Vv2 =0，uU-VvV= 1')%regarding uv as unknowns and a as a parameter 
uU= [1/2/(a + 1)*(-2*a + 2x*(-a)^(1/2)) +1]vV= [1/2/(a+ 1)*(-2*a + 2*(-a)^(1/2))] 


[1/2/(a + 1)*(-2*a - 2*(-a)^(1/2)) + 1] [1/2/(a + 1)*(-2*a - 2*(-a)^(1/2))] 
>>[a,uU] = Solve('ax*u'*2 + Vv^2'，uU-vV= 1，au') s%regards only v as a parameter 
a= -vc*2/(v*2 + 2x*v+ 1) U=Vvr+1 


Note that in the case where the routine“solve()”finds the Symbols more 
than the equations in its input arguments 一 Say, AM Symbols and N equations with 
上 > N 一 itregards the N Symbols closest alphabetically to “x” as variables and 
the other M -- N Symbols as constants, giving the priority of being a variable to 
the Symbol after “x ”than to one before “x"” for two Symbols that are at the Same 
distance from “x". Consequently, the Priority order of being treated as a Symbolic 
variable ls as follows: 


X>Yy>W>Z>V>U>t>S>Pm>9>… 


Actually，we can use the MAILAB built-in function“findsym()”to see the 
priority order. 


>>SymSs XyYZdqrstuvw sdeclare 10 Symbols to consider 
>>findsym(X +yY + Zrdqrr +S+ tr*u -vv - w10) %symbolic variables? 
ans = XiyiWzivuit sryd 


4.8 AREAL-WORLD PROBLEM 


Let's see the following example. 


卫 xXample 4.3. The Orbit of NASA'S “Wind” 9atellite. One of the previous NASA 
plans ls to launch a satellite，called Wind，which is to stay at a fixed position 
along a line from the earth to the sun as depicted in Fig. 4.7 so that the solar 
wind passes around the satellite on its way to earth. In order to find the distance 
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G=6.67x10-11 
Ms。= 1.98 x 1030[kg] 
Me。 = 5.98 x 1024[kgl 
m=the mass of satellite [kg] 
甩 = 1.49 x 1011[m] 
7=the distance of satellite from 













Sun [m] 
T= 3.15576 x 107[sec] 


=27/T 


S: Satellite , 
E: earth 。 








Figure 4.7 The orbit of a satellite. 


of the satellite from earth, we set up the following equation based on the related 
physical laws as 


AI.17 AIe11 2 2 
二 一 0 | 一 一 一 一 | 一 ro 一 0 (E4.3.1) 





(a) This might be solved for r by using the Cnonlinear) equation Solvers like 
the Toutine “newtons ()”(Section 4.0) or the MAILAB built-in routine 
“fsolve() .We define this residual error function (whose zero is to be 
found) in the M-file named“phys.m”and run the statements in the fol- 
lowing program“nm4e03 .m”as 


X0 1e6; %the initial (Starting) guess 

rn newtons('phys',Xx0,1e-4;,100) 点 newtons () 

rfs = fsolve('phys',x0,optimset('fsolve')) 5 fsSolve() 

rfs1 = fsolve('phys',x0,optimset('MaxFunEvals' ,1000)) %more iterations 
x0o01 = 1e10 %wWith another Starting guess closer to the Solution 

rfs2 = fsolve('phys',Xx01,optimset(' MaxFunEvals ,1000) ) 

residual_errs = phys([rn rfs rfs1 rfs2]) 


which yields 


rn = 1.4762e+011 <with residual error of -1.8908e-016> 
rfs = 5.6811e+007 <With_ residual error of 4.0919e+004> 
rfs1 = 2.1610e+009 <wWith residual error of 2.8280e+001> 
rfs2 = 1.0000e+010 <with residual error of 1.3203e+000> 


It seemas that, even with the increased number of function evaluations and 
another initial guess as Suggested in the warning message，fsolve() ”is 
not So Successful as “newtons() ”in this case. 
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Noting that Eq. (E4.3.1) may cause “division-by-zero ，we multiply both 
sides of the equation by 7( 尺 一 门 ” to rewrite it as 


13( 有 R 一 站 oo 一 GHMYCR -rr) +TGMr =0 (E4.3.2) 


We define this reslidual error function in the M-file named“physb.m” and 
run the following statements in the program“nm4e03 .m: 


rnb = newtons(' physb ' ,X0) 
rfsb = fsolve('physb',Xx0,optimset( ' fsolve ')) 
residual_errs = phys([rnb rfsb]) 


which yields 


rnb 1.4762e+011 <With residual error of 4.3368e-018> 
rfsb = 1.4762e+011 <Wwith residual error of 4.3368e-018> 


Both of the two routines “newtons() ”and "fsolve()” benefited from the 
function conversion and Succeeded in finding the Solution.， 


The results obtained in (a) and (b) imply that the performance of the non- 
linear equation Solvers may depend on the shape of the (residual error) 
function whose zero they aim to find. Here，we try applying them with 
scaling. On the assumption that the solution ls known to be on the order 
of 1011，we divide the unknown variable r by 1011 to scale it down into 
the order of one. This can be done by substituting = ”/101 into the 
equations and multiplying the resulting solution by 101. We can run the 
following statements in the program“nm4e03 .m”: 


Scale = 1e11; 

rns = newtons(' phys' ,x0/Sscale,1l1e-6,100,Scale)*Sscale 

rfss = fsolve(' phys' ,x0/scale,optimset( ' fsolve' ),Scale)*Sscale 
residual_errs = phys([rns rfss]) 


which yields 


rns 1.4762e+011 <With residual error of -6.4185e-016> 
rfss = 1.4763e+011 <with residual error of -3.3365e-006> 


Compared with the results with no scaling obtained in (a)，the routine 
fsolve() ”benefited from scaling and Succeeded in finding the solution. 
(cf) This example implies the following tips for solving nonlinear equations. 

。 If you have Some preliminary knowledge about the approximate value of 
the true Solution，scale the unknown variable up/down to around one and 
then Scale the resulting Solution back down/up to get the solution to the 
original equation. 

。 It might be better for you to apply at least two methods to solve the 
equations as a croSS-check. It is suggested to use “newtons () ”together with 
“fsolve() "for confirming the Solution of a System of nonlinear equations. 
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%nm4e03 一 astrophysics 
Clear，Clf 
global G Ms Me R T 
G = 6.67e11; Ms = 1.98e30; Me = 5.98e24; 
= 1.49e11; T = 3.15576e7; W= 2*pi/T; 
Xx0 = 1e6 s%initial guess 
format Short e 
disp('(a) ') 
rn = newtons(' phys ,Xx0) 
fs = fsolve('phys',x0 ,Optimset(' fsolve ')) 
fsolve('phys' ,x0)/fsolve('phys' ,x0,foptions) in MATLAB 5.x version 
fs1=fsolve( phys' ,x0,optimset('` MaxFunEvals' ,1000)) s%more iterations 
options([2 3 14])=[1e-4 1e-4 1000]; 
fsolve('phys' ,x0,options) in MATLAB 5.X 
01 = 1e10; %with another Starting guess Closer to the Solution 
fs2 = fsolve(' phys' ,x01,optimset(' MaxFunEvals' ,1000) ) 
esidual_errs = phys([rn rfs rfs1 rfs2]) 
isp( (b) ) 
nb = newtons('physb ' ,Xx0) 
rfsb = fsolve('physb',x0,optimset( fsolve ' )) 
residual _errs = phys([rnb rfsb]) 
disp('(c) ') 
Scale = 1e11; 
rns = newtons( 'phys' ,x0/scale,1e-6,100,Sscale)*scalei 
rfss = fsolve('phys',x0/Sscale,optimset('fsolve' ) ,Scale)*Scale 
residual _errs = phys([rns rfss]) 





function f = phys(x,Sscale) ; 

if nargin < 2，Scale = 1;j end 

global G Ms Me R T 

WwW= 2x*pi/T; Xx = X*Scalejf= Gr(Ms/(x.^2 + eps) - Me./((R - X).^2 + eps))-xx*w^2; 


function f = physb(x,Scale); 

if nargin < 2，Scale = 1; end 

global G Ms Me R T 

WwW= 2*pi/T; X = X*SCcalejf= (R-X).^2.*(wW^2*Xx.^3 - G*Ms) + Gx*Me*xXx.^2; 
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4.1 Fixed-Point Iterative Method 


Consider the Simple nonlinear equation 
ro)=x 一 3x+1=0 (P4.1.1) 
Knowing that this equation has two roots 


xX2 一 1.5 土 V1.25 六 2.6180 or 0.382; xx?1 写 0.382，x2” 写 2.6180 
(P4.1.2) 
investigate the practicability of the fxed-point iteration. 


(a) First conslider the following iterative formula: 


1 
xl 一 8uCxk) 一 本 G 十 1]) (P4.1.3) 
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Figure P4.1 Iterative method based on the fixed-point theorem. 
Noting that the first derivative of this iterative function gn(xz) ls 
2 
Sa(X) 一 了 5 (P4.1.4) 
determine which solution attracts this lteration and certify it in Fig. 
P4.1a. In addition，run the MAILAB routine“fixpt()”to perform 


the iteration (P4.1.3) with the initial points xo 一 0, x0 一 2, and xo 一 3. 
What does the routine yield for each initial point? 


(b) Now consider the following iterative formula: 


1 
Xi 一 SpO) 一 3 一 一 (P4.1.9) 
大 


Noting that the first derivative of this iterative function gp(X) is 
] 
8 (X) 一 和 (P4.1.0) 


determine which solution attracts this iteration and certify it in Fig. P4.1b. 

In addition, run the MAILAB routine“fixpt()”to carry out the itera- 

tion (P4.1.3) with the initial points xo 一 0.2, xzo 一 1 and xo 一 3. What 

does the routine yield for each initial point? 

(cf) This illustrates that the outcome of an algorithm may depend on the Start- 
jing point， 
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4.2 Bisection Method and Fixed-Point Iteration 


Consider the nonlinear equation treated in Example 4.2. 
jx) =tan(r 一 xz) 一 YX 一 0 (P4.2.1) 


Two graphical solutions of this equation are depicted in Fig. P4.2，which 
can be obtained by typing the following statements into the MAILAB 
command window: 


>>ezplot( 'tan(pi-x)',-pi/2,3*pi/2) 
>>hold on，ezplot('x+0'，-pi/2,3*pi/2) 


(a) In order to use the bisection method for finding the solution between 
1.3 and 3, Charley typed the statements Shown below. Could he get the 
right solution?7 If not, explain him why he failed and suggest him how 
to make it. 


>>fp42 = inline( tan(pI-x)-X' ，X'); 
>>TolX = 1e-4; MaxIter = 50; 
>>X = bisct(fp42,1.5,3,TolLX,MaxIter) 


(b) In order to find some interval to which the bisection method is applica- 
ble, Jessica used the MAILAB command “find()”as shown below. 
>>X = [0: 0.5: pil]jy= tan(pi-x) - X; 
>>Kk = find(y(1:end-1).*y(2:end) < 0); 
>>[X(k) xX(k + 1); y(k) y(k + 1)] 
ans = 1.5000 2.0000 2.0000 2.5000 


-15.6014 0.1850 0.1850 -1.7530 


This Shows that the Sign of jx) changes between xx 一 1.3 and 2.0 
and also between YY 一 2.0 and 2.3. Noting this, Jessica thought that she 
might use the bisection method to find a solution between 1.5 and 2.0 
by typing the following command. 


>>X=bisct(fp42,1.5,2,TolLX,MaxIter) 


Checkthe validity ofthe solution 一 that is, check if jx) = 0ornot 一 by 
typing 

>>fp42(X) 
H 开 her solution ls not good, explain the reason. If you are not Sure about 


it, you can try plotting the graph in Fig. P4.2 by typing the following 
statements into the MAILAB command window. 


>>Xx = [-pi/2+0.05:0.05:3x*pi/2 - 0.05]; 
>>p1lot(xtan(pIi - X),，X;X) 
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Figure P4.2 The graphical solutions of tan(r 一 X%) 一 X=0ortan(r 一 X) 一 X. 


(cf) This helps us understand why fzero(fp42,1.8) leads to the wrong Solu- 
tion even without any warning message as mentioned in Example 4.2. 


(c) In order to find the solution around x 一 2.0 by using the fxed-point 


iteration with the initial point xo 一 2.0, Vania defined the iterative func- 
tion as 





>>gp421 = inline('tan(pi - X)"，Xx'); 5%xX 一 SIC) 一 tan (区 X) 
andtyped the following statement into the MAILAB command window. 
>>X = fixpt(gp421,2,ToLX,MaxIter) 


Could she reach the solution near 2? Will it be better 寺 you Start the 
Ioutine with any different initial point? What 1S wrong? 


(d) Itha, seeing what Vania did, decided to try with another iterative formula 


tan-L1x 一 元 ， X 一 82(C) 一 元 一 tan-1x (P4.2.2) 


So She defined the iterative function as 
>>gp422 = ;inline('pi-atan(x)'，'x');%x 一 gg) = 工 一 tanrl0) 
andtyped the following statement into the MAILAB command window: 

>>X = fixpt(gp422,2,ToLX,MaxIter) 


What could she get? Is it the right solution? Does this command work 
with different inlitial value, like 0 or 06, which are far from the Solution 
we want to fnd? Describe the difference between Vania's approach and 
Itha's， 


4.3 


4.4 


4.S 
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Recursive (Self-Calling) Routine for Bisection Method 


Asstated in Section 1.3, MAITLAB allows us to make nested (Tecurslive) rou- 
tines which call itself. Modify the MAILAB routine “bisct()”(in Section 
4.2) into a nested routine“bisctF()”and run it to Solve Eq. (P4.2.1). 


Newton Method and Secant Method 


As can be Seen in Fig. 4.5, the secant method introduced in Section 4.5 
Was devised to remove the necessity of the derivative/gradient and improve 
the convergence. But, it Sometimes turns out to be worse than the Newton 
method. Apply the routines“newton()”and “secant()”to Solve 


记 4C0 一刀 一 刀 --x+T1L=0 (P4.4) 


starting with the initial point xo = 一 0.2 one time and xo = 一 0.3 for another 
Shot. 


Acceleration of Aitken-Steffensen Method 

A Sequence converging to a limit x2 can be described as 
太一 XI 一 etlSA4e 4 一 ) 

X2 一 KE 十 1 


with lim 
Koo X%2 一 X 


= 4(4| < D (P4.5.1) 


In order to think about how to improve the convergence Speed of this 
Sequence,， we define a new sequence PK as 


证 X2 一 XiK 2 
一 一 入 45 一 Co 一 了 HG 一 MD SC 一 其) 
X0 一 大 ( 一 无 尼 江 


2 妇 2 2 
(一 NAHIX 一 XI 十 MEHET SC 一 2 十 X 


2 
XKHIXK 1 一 XK 
Te (P4.5.2) 
XkH1L 一 2XK 十 2Xk -1 。 


(a) Check that the error of this Sequence Pk 18 as follows， 
了 HT 一 坏 
X 一 1 一 7 一 一 
XI 一 2XK 十 XK 1 

XIGEH 一 2 不 十 了 一 这 十 2x0 1 一 人 


XI 一 2XK 十 XK1 


一 xo 
到 (Coke 一 XTD)” 
Xk+HL 一 2XK 十 Xk-1 
(一 2 一) 十 (2 一 了 允 -D) 
一 (2 一 MkHD 十 202 一 了 ) 一 (2 一 了 -1 
(一 4 十 DGx29 一 六 TD 


一 YY2 一 了 一 一 一 一 一 一 一 一 一 0 P4.5.3 
和 2 


一 2 = 二 : 
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Table P4.5 Comparison of Various Methods Applied for Solving Nonlinear 
Equations 


有 区 区 了 区 于 
-| 

加 本 本 古语 本 四 

nx 


有 | 西 胡 丁丁 本 呈 二 本 加 引 现 大 末 
[3 本 天 本 卧 本末 丁 项 酉 台 本 2 
本 辆 本本 区 加 本 于 攻 本 本 


区 国 有 
| 二 


(cf) Since the flops() command is no longer available in MATLAB 6.x version， the numbers of 
foating-point operations are obtained from MAILAB 3.x version So that the readers can compare 
the various algorithms in terms of their computational loads. 





(b) Modify the routine “newton()”into a routine“stfns ()”that generates 
the Sequence (P4.3.2) and run ilt to Solve 


Jia) 一 tan(r 一 2z) 一 X 一 0 (with xzo 王 1.0)  (P4.5.4) 
万 = 刀 一 妆 -x+1=0 (withxo=0)  (P4.5.5) 
JJ=--54=0 (with xo 一 0)  (P4.5.6) 





Fill in Table P4.3 with the results and those obtained by using the 
TIoutines“newton()”,“secant()”(with the error tolerance TolLX 一 
10-5),“fzero()” and “fsolve()” 

4.6 Acceleration of Newton Method for Multiple Roots: Schroder Method 


In order to Improve the convergence Speed, Schroder modifies the Newton 


iterative algorithm (4.4.2) as 


1 Ji) 
六 Co 


with AM : the order of multiplicity of the root we want to find 





XK 二 1 一 XK 一 (P4.6.1) 


?9 


Based on this idea，modify the routine “newton()”into a routine 
“schroder()”and run it to Solve Eqs. (P4.5.4.6). Fill in the corresponding 
blanks of Table P4.3 with the results. 
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4.7 Newton Method for Systems of Nonlinear Equations 


Apply the routine“newtons()”(Section 4.60) and the MAILILAB built-in 
Ioutine “fsolve()”(with [x0 y0] = [1 0.3]) to solve the following Systems 
of equations. Fill in Iable P4.7 with the results. 


2 
下 人 1 (P4.7.1) 
(b) S$cos6; + 6cos(6; 十 9) = 10 
Ssin0i 十 6sin(0 十 6) 一 4 (P4.7.2) 
(9 3z2+4 和 2 =3 
X2 十 7 二- V37/2 (P4.7.3) 
(d) 好 二 10x 一 2=5 
1 (P4.7.4) 
1 2 2 一 
(e) 刀 一 V3xy 十 272 = 10 (P4.7.5) 
4x2 十 3V3xy 十 yy 一 22 .7. 
SN 3 
2 2 二 (P4.7.6) 
2 2 二 
二 4 (P4.7.7) 
(DJ) xey 一 zx 二 > 一 3 
XY 十 y 二 tanx 一 siny 一 0 (P4.7.8) 
本 。 (P4.7.9) 
(P4.7.10) 


60x 一 180x2y 一 30xy 一 1 
4.8 Newton Method for Systems of Nonlinear Equations 


Apply the routine“newtons ()”(Section 4.6) and the MAILAB built-in 
Ioutine“fsolve()”(with [x0 y0 z0]=[1 1 JU)tosolvethe following 
Systems of equations. Fill in Tabple P4.8 with the results. 


(a) xyz 一 一 1 
xX2 十 272 十 4z2 一 7 (P4.8.1) 
2x 十 妇 十 6z 一 7 

(b) xyz 王 1 
x2 二 2y3 十 2 一 4 (P4.8.2) 


X 十 272 一 刀 一 2 
(c]) xz 十 42 十 9z2 一 34 


xX2 十 9 一 Sz 一 40 (P4.8.3) 
xz 一 一 7 

(d) xz 十 2sin(yr/2) 十 莹 一 0 
级 (P4.8.4) 


ex+7 一 zz 一 0 
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Table P4.7 Applying newtons () /fsolve() for Systems of Nonlinear Equations 


-一 一 王 一 一 区 


X0 一 

站 证 | 
一 
本 天 村 证 和 国生 生生 二 
Eee 
eol | 
一 
下 





~ | 
一 
PIT 


(P4.7.10) 上 1 | oo | | oo | lle-160) 


(cf) The numbers of foating-point operations and the residual (mismatching) errors in the parentheses are obtained 
from MATLAB 53.x version. 








PROBLEMS 205 


Table P4.8 Applying newtons ( )fsolve() for Systems of Nonlinear Equations 


一 
(P4.8.D IFooll | 11102e-16 (1.1102e-16) 0 


X0 一 [1 


(P4.8.2) 


X0 一 [1 


6011 4735 


(P4.8.4) IFGColl | 4.5506e-15 (4.6576e-15) | 


X0o 一 [1 


(e) xz 十 闪 十 z 一 14 
X2 十 2 六 一 z 一 6 (P4.8.5) 
X 一 3 十 刀 一 一 2 

(人 xz 一 127y 十 z 民 一 5 
3x2 十 妇 一 2z 一 7 (P4.8.0) 
x 十 24 并 一 2sin(rz/18) = 25 
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(g) 六 十 太一 2z 一 6 
xX2 一 27 十 z 一 3 (P4.8.7) 
2xz 一 3 六 一 寻 = 一 27 


Newton Method for a System of Nonlinear Equations with Varying Para- 
meter(S) 

In order to find the average modulation order xi for each user of an OFDM 
(orthogonal frequency division multiplex) System that has N(128) Subcha- 
nnels to assign to each of the four users in the environment of nolise power 
No and the bit error rate (probability of bit error) 已 ，a communication 
System expert，Mi-hyun，formulated the problem into the System of five 
nonlinear equations as follows: 


Po0 = (2200n2 一 1 十 D 字 2(erfe-K(P/27 一 入 =0 (P4.9.1) 
for 一 1 2,3,4 
4 

广 oJ= 》、 二 -~-N=0 (P4.9.2) 


i1 


where N =128 and wi is the data rate of each user 


where erfc-1(x) is the inverse function of the complementary error function 


2 RS 2 5 
erfc(x) = 一 e di 一 1 一 去 上 e 一 di 一 1 一 erf(x) (P4.9.3) 
VT ,0 


and defined as the MAITLAB built-in function “erfcinv() .She defined 
the mismatching error (vector) function as below and Save it in the Mr-file 
named “fp_bits .m”. 


function y = fp_bits(x,aPe) 
%x(i),LI= 1:4 correspond to the modulation order of each User 
%X(5) corresponds to the Lagrange multiplier (Lambda) 
if nargin < 3，Pe = 1e-4; 
if nargin < 2，a = [64 64 64 64]; end 


end 

N = 128; NO = 1; 

Xx14 = X(1:4) 

y = (2.^x14.*(10g(2)*x14 - 1)+1)*NO/3*2x*erfcinv(Pe/2).^2 - X(5); 
y(5) = Sum(a./x14) - Ni 





Compose a program which solves the above System of nonlinear equations 
(with No = 1 and 己 = 10-) to get the modulation order xi of each user 


4.10 


4.11 
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for five different sets of data rates 
a 一 [32 32 32 32], [64 32 32 32], [128 32 32 32], [256 32 32 32]，and [$12 32 32 32] 


and plots al1/X1 (the number of Subchannels assligned to user 1) versus al 
(the data rate of user ]1). 


Temperature Rising from Heat Flux in a Semi-infinite Slab 


Consider a Semi-infinite Slab whose temperature Tises as a function of posi- 
tionXy > 0 andtime 上 > 0 as 


TO 1D) = cG(2 erfc(s) |] with 光一 /4at (P4.10.1) 
大 VTS 


where the function erfc(O is defined by Eq. (P4.9.3) and 


CO (heat fux) 一 200 Jjm2s， 有 (conductivity) = 0.015 JJ/m/sAC， 
a (diffusivity) = 2.5 x 10- m2/s 


In order to find the heat transfer Speed,，a heating System expert, 开 yung- 
won，wants to Solve the above equation to get the positions x(1) with a 
temperature rise of 了 = 30"C at 一 10:10:200 S. Compose the program 
which does this job and plots x(f) versus 1 


Damped Newton Method for a Set of Nonlinear Equations 


Consider the routine“newtons ()”,， which is made for solving a System of 

edquations and introduced in Section 4.0. 

(a) Run the routine with the initial point (xlo, X20) 一 (0.3, 0.2) to Solve 
Eq. (4.60.5) and certify that it does not yield the right solution as depicted 
in Fig. 4.6c. 

(b) In order to keep the step Size adjusted in the case where the norm of the 
vector function fxki at iteration K 十 1is larger than that of fxx) at 
iteration 上, insert (activate) the statements numbered from 1l to 6 of the 
TIoutine “newtons ()”(Section 4.0) by deleting the comment mark (%) at 
the beginning of each line to make a modified routine“newtonds () ”， 
which Implements the damped Newton method. Run it with the initial 
point (xlo, x20) 一 (0.3, 0.2) to Solve Eq. (4.6.3) and certify that it yields 
the right solution as depicted in Fig. 4.6d. 

(c) Run the MAILAB built-in routine“fsolve()”with the initial point 
(xlo, X20) 一 (0.9, 0.2) to Solve Eq. (4.6.5). Does it present you a right 
Solution? 


NUMERICAL 
DIFFERENTIATION/ 
INTEGRATION 





5.1 DIFFERENCE APPROXIMATION FOR FIRST DERIVATIVE 


FEor a function 太 x) of a variable x, its first derivative is defined as 


六 oo = lim J7C 十 用 二 大) 


in ， (5.1.1) 


However, this gives our computers a headache，since they do not know how 
to take a limit. Any input number given to computers must be a definite num- 
ber and can be neither too Small nor too large to be understood by the com- 
puter. The “theoretically” infinitesimal number invyolved in this equation ls a 
problem. 

A Simple approximation that computers might be happy with ls the forward 
difference approximation 


J 十 /一 /xc) 


Droc,7) 一 下 


(P is step Size) (5.1.2) 


How far away is this approximation from the true value of (5.1.1)? In order to do 
the error analysis，we take the Taylor series expansion of 太 (x 十 凡 ) about X as 


7 万 3 
JG 十 站 = Ge) 二 APGCo 十 7 (0 十 了 太 (Or0) 十 …， (5.1.3) 


4PPlied Nu1aerical Metpods Using MA4774B9， by Yang,，Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 人 儿 Sons, Inc. 
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Subtracting jx) from both sides and dividing both Sides by the step Size 六 yields 


J 十] 一/C) 
忆 


= 二 OOD (3.1.4) 


/ (2) 包 G) 
Drilx, 月 = 的 二 二 


where O(g(P)), called “big Oh of g(P) ,denotes a truncation error term Propor- 
tional to g(P) for Im < 1. This means that the error of the forward difference 
approximation (3.1.2) of the first derivative ls proportional to the Step Size 灵 ，or， 
equivalently, in the order of 凡 ]. 

Now, in order to derive another approximation formula for the first derivative 
having a Smaller error, let's remove the first-order term with respect to 六 from 
Eq. (3.1.4) by Substituting 217 for in the equation 


FFC 十 21) 一 Fr) 472 


/ 和 2 .oO) G) 
站 Fo, 21) 三 7 = 了 (00 十 本 上 5 (C) 十 …: 


and subtracting this result from two times the equation. Then，we get 


jx 十 从 一 /ax) Ac 十 20) 一 Jo) 


2 万 ,万 ) 一 万 ,21) 一 2 
FPC 7) Fl 21) 


艺 
= 记 CD) 一 了 Jo 区 
2DFl1(0x， 刀 ) < Drlix， 21) 
2 一 1 
一 e+2 人 二 4F 十 站 一 3700 
27 
GD) 二 O(0) (5.1.5) 


刀 P2(, 万 ) 一 


which can be regarded as an improvement over Eq. (3.1.4)，Ssince it has the 
truncation error of O(P22) for |P < |. 
How about the backward difference approximation'? 


Dai 站 二 二 = Drio, -有 (isstep size) (5.1.6) 


This also has an error of O(P) and can be processed to yield an improved version 
having a truncation error of O(1). 


2Dpol0j) 一 Do ,21) 38) 一 41x 一 站 十 je 一 27) 
让 本 27 
= 六 CD)+TOCO) (5.1.7) 


Di,7) 三 


In order to derive another approximation formula for the first derivative，we 
take the Taylor series expansion of 太 xz 十 岂 ) and jx 一 六 ) uptothe fifth order 
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to write 





12 1 4 14 4 15 .4 
Je+ 月 =JOD0TMGD+ 本 1 可 J19G0 二 而 1 二 可 10D 二 … 














玉 万 和 ro) 刀 rog) 包 ra 刀 ro 
了 攻 一 朋 二 三 局 的 林寺 二 一 有 网 卡 克 ) 一 本 (下 


and divide the difference between these two equations by 217 to get the central 
difference approximation for the first derivative as 


X 十 玉 ) 一 太 (x 一 天 用 2 4 
大 人 关 下 
27 3 9! 
= 7ooD + O(0D) (5.1.8) 
which has an error of O(P2) similarly to Eqs. (3$.1.5) and (5.1.7). This can also be 
processed to yield an improved version having a truncation error of O (14). 


je 上 月-JG 一 月 Je+2 癌 一 FGc 一 2 


22Dc, 门 一 Dear,21) 一 4 
2(X， 思 ) 2(0x， 27) 7 让 


1274 
=31 -19 一 … 


22De 0 有 一 Dai(x, 2 和 


Da, 思 = 5 
8JGC TD 一 8jG 一 月 一 Je 二 2 十 Ac 一 2 有 
本 127 
= Fo) 十 OO) (3.1.9) 


Furthermore, this procedure can be formularized into a general formula, called 
“Richardson's extrapolation , for Improving the difference approximation of the 
derivatives as follows: 


<Richardson s extrapolation> 
27Djnx,7) 一 Fnx,27) 
22 一] 
2Dpr0x1) 一 Dopn(c,21) 
22 一 1 
2 Doom 一 Den(r，27) 
2 和 2 一 1 


Drn+lc,7) 一 


(P: the order of error) (9.1.10a) 





DopnHiGc 站 一 (5.1.10b) 


De2o+D(GCc 7 一 ($.1.10c) 





5.2 APPROXIMATION ERROR OF FIRST DERIVATIVE 


In the previous section，we derived Some difference approximation formulas 
for the first derivative.， Since their errors are proportional to Some power of 
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the step-Size 1, it Seemas that the errors continue to decrease as 几 gets Smaller. 
However, this ls only half of the story since we considered only the truncation 
error caused by truncating the high-order terms in the Taylor series expansion 
and did not take account of the round-off error caused by quantization. 

In this section，we will discuss the round-off error as well as the truncation 
error So as to gain a better understanding of how the compnuter really works. For 
this purpose, Suppose that the function values 


J(Cxz 十 21)， Jx 十 7])， J(z)， J(x 一 帮 )， JJ 一 21) 
are quantized (rounded-off) to 


) 三 jx 十 21) 十 e2， y1 = 十] 十 el 
y0 三 Jox) 十 eo (5.2.1) 
y_1=Jx 一 由 十 e-1，， yy =Jx 一 21) 十 e- 
where the masgnitudes of the round-offt (quantization) errors ez, el, eo, e_l, and 


e_2 are all smaller than some positive number s, that is, |ei| 和 s. Then, the total 
error of the forward difference approximation (3.1.4) can be derived as 


7yI 一 Xe 十 有 十 el 一 Jr) 一 eo G.1-0 vv el 一 e0 天 
万 万 ) = 一 一 一 二 一- 一 ea 
7 月 一 一 六 7 二 + 
一 天 多 天 
IDriec,P) 一 万 ()| 去 一 二 < WwWith 天 一 FO(Cx) 








Look at the right-hand Side of this inequajlity 一 that ls, the upper bound of error. 
It consists of two parts; the first one ls due to the round-off error and in inverse 
proportion to the step-Size 由 ， while the second one ls due to the truncation error 
and in direct proportion to 17. Therefore, the upper bound of the total error can 
be minimized with respect to the step-Sslize 1 to give the optimum Step-Size PP。as 


dd 12s | 天 28 | 天 1| 2 
-| 一 十 一 一 四 三 三 二 二 二 一. 二 :0; 1 = 2 /| 一 一 .22 
所 ( 直 + 2 ) 现下 1R 人 


Thetotal error of the central difference approximation (3.1.8) can also be derived 
as follows: 


一 克 一 一 记 ) 一 e_- 
区 5 二 洱 7 7CT 有 十 e 二 Ja 一 月 一 e-l 

27 27 

(5.1.8) el 一 e_1 天 2 ，” 

一 一 一 一 一 一刀 
Co) 十 有 十 

/ El1 一 6-1 | 天 1| 2 28 | 开 ?| 2 (3) 
1DeGxz,P 一 三 (xl 2 | 于 坊 十 -大 With 大 2 矿 (GD) 
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The rightrhand side of this inequality ls minimized to yield the optimum step 
SiZe 1 。 as 


ae s 1 有 | ,) 3e 
2 一 几 一 一 一 一 1 一 1 RS -一 一 223 
矶 (3 


Similarly,， we can derive the total error of the central difference approximation 
(9.1.9) as 








8 8el 一 8e_1 一 ez 十 e-?| | 天 4 4 
站 4(x, 太 ) 一 二 | 一 一 一 一 必 
1DPa0o 7 一 三 (| 1 芒 十 3 
18e Ka4| ， | 
站 也 KR 一 FG) 
0 | 


and find out the optimum step Size 及。 as 


d 13s |K4| ， 38 2IK4| ,| 45e 
人 ja=0， 帮 = 5.2.4 
d1 ( 芭 并 有 5 了 4| 有 | ee 


From what we have Seen So far, we can tell that, as we make the step Size 灵 
smaller, the round-off error may increase，while the truncation error decreaseS. 
This ls called “step-Size dilemma'`. Therefore, there must be Some optimal step 
Size 1。 for the difference approximation formulas，as derived analytically in 
Eqs. (9.2.2), (9.2.3), and (3.2.4). However, these equations are only of theoretical 
value and cannot be used Practically to determine 1。because we usually don't 
have any information about the high-order derivatives and，consequently，we 
cannot estimate K1, K2,.... Besides, noting that Po。 minimizes not the real error， 
but its upper bound, we can never expect the true optimal Step Size to be uniform 
for all x even with the Same approximation formula. 

Now, we can verify the step-Slize dilemma and the existence of some optimal 
Step Size 1。by computing the numerical derivative of a function，Say，j 太 (xz) 一 
SinX，whose analytical derivatives are well known. To see how the errors of the 
difference approximation formulas (9.1.4) and (3.1.8) depend on the Step Size 几 ， 
we computed their values for x = 王立 /4 together with their errors as Summarized 
in Iables 5$.1 and 3.2. From these results, it appears that the errors of (3.1.4) and 
(3.1.8) are minimized with 疡 久 10-8 and 7 久 10-5, respectively. This may be 
justifed by the following facts: 








。Noting that the number of Signifticant bits js 32, which is the number of man- 
tissa bjits (Section 1.2.1)，or, equivalently, the number of Significant digits 
is about 52 x 3/10 交 16 (since 210 s 103), and the value of Fox) = sinx is 
less than or edqual to one, the round-off error is roughly 


6 久 10-187/2 
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Table 5.1 The Forward Difference Approximation (5.1.4) for the First Derivative of fx = 
sinx and lts Error from the True Value (cosr/4 = 0.7071067812) Depending on the Step 


Sizen 
友 二 10 飞 


/1 三 0.1000000000 
12 三 0.0100000000 
13 三 0.0010000000 
14 三 0.0001000000 
15 三 0.0000100000 
16 三 0.0000010000 
17 三 0.0000001000 
18 三 0.0000000100* 
1 三 0.0000000010 
110 三 0.0000000001 


DIKz=r/4 


0.6706029729 
0.7035594917 
0.7067531100 
0.7070714247 
0.707103243506 
0.7071064277 
0.70710674354 
0.7071067842 
0.7071068175 
0.7071077057 


DPK 一 Dik-D 


0.0329565188 
0.0031936183 
0.0003183147 
0.0000318210 
0.0000031821 
0.0000003176 
0.0000000389 
0.0000000333* 
0.0000008882 


Diklx=r/4 一 COS(T/4) 
一 0.03650380828 
一 0.00354728950 
一 0.0003S$367121 
一 0.0000333506532 
一 0.000003535354 
一 0.00000035344 
一 0.00000003381 
0.00000000305S* 
0.00000003636 
0.00000092454 


1 = 0.0000000168 (the optimal value of 六 .obtained from Eq. (3.2.2)) 


Table 5.2 The Forward Difference Approximation (5.1.8) for the First Derivative of fog) = 
sinx and lts Error from the True Value (cosr/4 = 0.7071067812) Depending on the Step 


Sizen 
友 二 10 飞 


/1 三 0.1000000000 
/2 三 0.0100000000 
13 三 0.0010000000 
14 三 0.0001000000 
15 三 0.0000100000” 
16 三 0.0000010000 
17 三 0.0000001000 
18 三 0.0000000100 
1 三 0.0000000010 
10 三 0.0000000001 


Da2klxz=r/4 


0.7059288590 
0.7070949901 
0.7071066633 
0.7071067800 
0.7071067812 
0.7071067812 
0.7071067804 
0.7071067842 
0.7071067620 
0.7071071506 


Dx 一 Dok-D 


0.00116061371 
0.0000116672 
0.0000001167 
0.0000000012 
0.0000000001* 


一 0.0000000009 


0.0000000039 


一 0.0000000222 


0.0000003886 


Daoklr=r/4 一 COS(T/4) 


一 0.00117792219 
一 0.00001178505 
一 0.00000011785 
一 0.00000000118 
一 0.0000000000] 
0.00000000005 
一 0.00000000084 
0.00000000305 
一 0.00000001915 
0.00000030942 


1 = 0.0000059640 (the optimal value of 六 .obtained from Eq. (3.2.3)) 


。Accordingly, Eqs. (53.2.2) and (9.2.3) give the theoretical optimal values of 
Step Size 灵 as 


E 10- 107/2 8 
入 二 2/ 启 =2 = 1.68 x 10 
人 VTFGC7 人 [二 sa 
62 
j 让 ER 


APPROXIMATION ERROR OF FIRST DERIVATIVE 215 


























100 
10-5 
10-4 

_8| ho:optimal value 了 | hu: optimal value |! 

8 O O 
由 雪人 | 5 去 2s | 2 
DH(X 月 -FoO|< 人 + De2(% 月 -fo0|<2 + 
! 二 
1010 h 100 10 0 ho hm 100 
(a) Error bound of Eq. (5.1.4) vs. step size (b) Error bound of Eq. (5.1.8) vs. step size 


Figure 5.1 Forward/central difference approximation error of first derivative versus step size /. 


Figure 3.1a/b Shows how the error bounds of the difference approximations 
(3.1.4)/(5.1.8) for the first derivative vary with the step-Slize 1, implying that there 
is Some optimal value of step-Slize 刀 with which the error bound of the numerical 
derivative ls minimized. It seems that we might be able to get the optimal step- 
Size 1 by using this kind of graph or directly using Eq. (3.2.2),(3$.2.3) Or (3.2.4). 
But as mentioned before, it is not possible, as long as the high-order derivatives 
are unknown (as is usually the case). Very fortunately，Tables 5.1 and 5.2 sug- 
gest that we might be able to guess the good value of /by watching how 
Small | 关 K 一 DiCDl is for a given problem. On the other hand，Fig. 5.23a/b 
Shows the tangential lines based on the forward/central difference approximations 
(93.1.4)/(3.1.8) of the first derivative at X 王立 /4 with the three values of step- 
Size 1. They imply that there is Some optimal step-Slze 17。and the numerical 
approximation error becomes larger 寺 we make the step-Sslize ] larger or Smaller 
than the value. 




















1 上 1 
0.8 上 上 0.8 
0.6 上 上 0.6 
0.4 上 0.4 
0.2h L] ， ， 0.2 








| | | | 
0 0.5 1 15 X 2 0 0.5 1 15 X 2 
(a) Forward difference approximation by Eq. (5.1.4) (b) Central difference approximation by Eq. (5.1.8) 


Figure 5.2 Forward/central difference approximation of first derivative of fg = Sin X. 
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5.3 DIFFERENCE APPROXIMATION FOR SECOND 
AND HIGHER DERIVATIVE 


In order to obtain an approximation formula for the second derivative，we take 
the Taylor series expansion of jx 十] and 太 xz 一 站) uptothe fifth order to 
Write 





万 2 帮 3 万 4 75 
JG+ 月 =JGO+MGO+JOOD+ 林 Jo 页 7100+ 林 GOCD 二 … 














万 PP 和 ro) 好 rog) 和 rqg 和 ro 
JjG 一 有 = 一 风干 太 人 可 的 十 而 太 一 可 太 0 十 


Adding these two equations (to remove the (xz) terms) and then subtracting 
27F(xz) from both sides and dividing both sides by 12 yields the central difference 
approximation for the second derivative as 


同 汪 者 一 
po ,月 = 人 
7 211 
二 网 下 责 六 网 二 有 全 3) 


which has a truncation error of O(p). 
Richardson'`s extrapolation can be used for manipulating this _ equation to 


remove the 12 term， which yields an improved version 


22DO0 人 -Do0,210 -ae+21+16Fc+ 站 -30100+16Fc 一 站 -Fe 一 2 站 
22 一 1272 








2 有 G) 人 
二 二 /的 + 


六 凡 测 法 一 Fe +21 二 16F 十 站 -30fo0) 二 16F 一 站 -Fe 一 21) 
12722 


= fooD + Oo) (5.3.2) 


which has a truncation error of O(P4). 

The difference approximation formulas for the first and Second derivatives 
derived so far are Summarized in Table 3.3，where the following notations are 
Used: 


D7i) / DC / D 人 和) ls the forward/backward/central difference approximation for 
the Nth derivative having an error of O(1)(P is the step size) 


大 三 jc 十 人) 
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Now, weturn our attention to the high-order derivatives. But, instead of deriv- 
ing the Specific formulas, letfs make an algorithm to generate whatever difference 
approximation formula we want. For instance, 寺 we want to get the approxima- 
tion formula of the second derivative based on the function values 户 , 万 , 加 广 1， 
and 广 ?，we wiite 


cz 户 十 cl 万 十 coj+c 广 1 十 c-2 广 2 


恩人 大 而 三 


(3.3) 


and take the Taylor series expanslion of 户 , 万, 广 1 and 三? excluding Jon the 
Tight-hand side of this equation to rewrite it as 


DC 站 


2])” 21 2j)4 
co( +2M+ 下 2 二 + 





十 Cl ( 十 天 j 十 一 所 7 有 人 生 1p 二 】 十 co 万 


了 
He 
(2])” ao G (21 da 
， 外 这 和 
(cz 十 cl 十 co 十 c-1 十 c-?)j 十 jc 十 cl 一 c-1 一 2c_2) 乒 
人 
ER ER 9 2 
了 
= 一 23 1 1 22 (5.3.4) 
万 2 十 13 (3 十 3 引 191 3 引 一 1 Se 3 ) 


We Should solve the following set of equations to determine the _ coefficients 
Cc2, Cl, co, C_1，and c_ > So as to make the expression conform to the second 
derivative 思 2) atx 十 OP 一 2 


1 1 1 1 1 0 
2 1 0 -1 -2 本 0 
22/2! 1/21 0 1/24 22/21 co |=|1 (5.3.5) 
23/3! 1/3! 0 -1/3! -23/31 | | c_ 0 
24/4! 1/44 0 14 24/41 0 
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Table 5.3 The Difference Approximation Formulas for the First and Second Derivatives 


O() forward difference approximation for the first derivative: 


DriG 有 一 一 儿 (5.1.4) 


O(12) forward difference approximation for the first derivative: 


2DHio,7) 一 DHx,21) 于 一 记 十 和 7 二 3 方 





刀 12(x,7) 一 .1.5 
72(0x, 帮 ) 5 7 《 ) 
O() backward difference approximation for the first derivative: 
二 一己 G5.1.6) 
O(1p2) backward difference approximation for the first derivative: 
2Dpl0,1) 一 Do 21) 3 思 一 4 广 ! 十 广 
Di1) = 一 = 一 .1.7 
p(0c, 月 7 可 (5.1.7) 
O(Pp2) central difference approximation for the first derivative: 
太 二 : 广 1 
De,1j) = 一 一 .1.8 
200， 7) 7 (3.1.8) 
O(p9) forward difference approximation for the first derivative: 
22Do0c, 站 一 Dee,27) 户 十 8 刀 一 8 广 ! 十 广 ? 
Da40j = 一 = 一 (5.1.9 
0 獒 二 1 1 人 
O(12) central difference approximation for the second derivative: 
一 2 和 
放生 几 二 万， (5.3.1) 


O(Pp) forward difference approximation for the second derivative: 





全 22D20 站 -Doc,21 一 户 二 16 方 -30 力 +16 广 一 广 ? 
生生 
22 一 ] 127 
(5.3.2) 
O(12) central difference approximation for the fourth derivative: 





广 2 一 4 广 ! 十 6 四 一 4 万 十 户 


ocD= 


(from difapx(4,[-2 2]) (5.3.6) 
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function [cerr,eoh,A,b] = difapx(N,points) 
%5difapx.m to get the difference approximation for the Nth derivative 
1 = max(points); 
L = abs(points(1)-points(2))+ 1; 
ifL<N+1，error('More points are needed! ); end 
上 
1; 
2:L+2，Almin) =Am- 1,n)*l/(m - 1); end s%5Eq.(5.3.5) 


zeros(L,1); bIN+ 1) = 1 
C =(A(1:L,:)\b)'; s%coefficients of difference approximation formu1a 
err =A(L + 1,:)*c'j eoh =L-Ni s#%coefficient & order of error term 
if abs(err) < eps，err = A(L + 2,:)*c' jeohn=L -N+ 1; end 
if points(1) < points(2)，Cc = fliplr(c); end 





The procedure of setting up this equation and solving it ls cast into the 
MAILAB routine“difapx()”，which can be used to generate the coefficients 
of，Ssay,， the approximation formulas (3.1.7)，(3.1.9)，and (3.3.2) just for prac- 
tice/verification/fun，whatever your purpose is， 


>>format rat %to make all numbers represented in rational form 
>>difapx(1,[0 -2]) slst derivative based on {fo,Ff_i,F 2>} 
ans = 3/2 -2 1/2 %5Eq.(5.1-7) 
>>difapx(1,[-2 2]) %lst derivative based on {f_2,F_i,Tfo,Fi,Fz} 
ans = 1/12 -2/3 0 2/3 -1/12 %5Eq.(5.1.9) 
>>difapx(2,[2 -2]) %2n"d derivative based on {fz,fi,fo,Ff1i,F 2 } 
ans = -1/12 4/3 -5/2 4/3 -1/12 %5Eq.(5.3.2) 


上 Example S$.1. Numerical/Symbolic Differentiation for Taylor Series Expansion. 
Consider how to use MAILAB to get the Taylor series expansion of a func- 
tion 一 Say, e about x 一 0 一 which we already know js 


一 1 一 X 十 二 扩 一 一 大 一 X 一 一世 … 友 因 
2 3 有 
As a numerical method，we can use the MATLAB routine“difapx()” On 
the other hand，we can also use the MATILAB command“taylor()”，which 
is a Symbolic approach. Readers may put “help taylor”into the MAILAB 
command window to See its usage,， which ls Testated below. 





。 taylor(f) gives the fifth-order Maclaurin Series expanslion of 下 . 


。taylor(f,n + 1) with an integer n > 0 gives the 1th-order Maclaurin 
Series expanslon of 下 . 


。 taylor(f,a) with areal number(a) gives the fifth-order Taylor series expan- 
Sion of f about a. 
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。taylor(f,n + 1,a) gives the n th-order Taylor series expansion of f about 
default_variable = a. 


。 taylor(f,n + 1,aiy) gives the nth-order Taylor series expansion of f(y) 
about y = 


(cf) The target function f must be a legitimate expression given directly as the first 
input argument. 


(cf) Before using the command “taylor()”, one Should declare the arguments of the 
function as Symbols by putting the statement like “syms x 廿 ”. 

(cf) In the case where the function has Several arguments, it is a good practice to put the 
independent variable as the last input argument of "taylor()”, though taylor() 
takes one closest (alphabetically) to “x” as the independent variable by default only 
ifithas been declared as asymbolic variable and is contained as an input argument 
of the function f. 


(cf) One Should use the MATLAB command “sym2poly()” 迁 he wants to extract the 
coeffticients from the Taylor series expansion obtained as a Symbolic expresslion. 





The following MAILAB program “nm5e01”finds us the coefficients of fifth- 
order Taylor series expansion of e aboutx 一 0 by using the two methods. 


%nm5e01:Nth-order Taylor Series expansion for e^-Xx about xo in EX 5.1 
f=inline('exp(-xX) ，X' ); 
5; xo = 0; 
%Numerical computation method 
区 = feval(f,xo); 
h = 0.005 %5.01 or 0.001 make it worse 
tmp = 1; 
村 工 = 
mp 


1:N 

= tmpx*ixh;i %5i!(factorial 工 )*h“^ 工 

difapx(I,[-I II]); s%coefficient of numerical derivative 

2 = Crxfeval(f,xo + [-I:I]*h) ; 55/h^ 人 ii %dqerivative 
T(i+l) = dix/tmp; %5Taylor Series coefficient 

end 

format rat，Tn = fliplLr(T) %5dqescending order 

%Symbolic computation method 

Syms Xij TS = Sym2poly(taylor(exp(-Xx);N + 1,Xxo)) 

%discrepancy 

format Short，discrepancy=norm(Tn - Ts) 





5.4 INTERPOLATING POLYNOMIAL AND NUMERICAL 
DIFFERENTIAL 


The difference approximation formulas derived in the previous Sections are appli- 
cable only when the tarsget function jx) to differentiate is Somehow given. In 
this Section，we think about how to get the numerical derivatives when we are 


INTERPOLATING POLYNOMIAL AND NUMERICAL DIFFERENTIAL 221 


given only the data file containing Several data points. A_ possible measure is 
to make the interpolating function by using one of the methods explained in 
Chapter 3 and get the derivative of the interpolating function. 

Eor Simplicity, let's reconsider the problem of fnding the derivative of 太 (x) 一 
Sinx at X 一 和 /4，where the function is given as one of the following data 
point Sets: 


| 人) ( 到 sm 芝 ) 
|osano (可 本) (了 (到 ) (全 村 ]| 


2 元 ”27 3  ， 3 4 关 ， 4 5 .Sr 067r .6 
一 ,Sn 一 ,| 一 ,Sn 一 | ,| 一 ,Sn 一 | ,| 一 ,Sn 一 | ,| 一 ,Sn 一 
16 16 16 16 16 16 16 16 16 16 


We make the MAILAB program“nm540”， which uses the routine “lagranp()” 
to fnd the interpolating polynomial, uses the routine “polyder() ”to differentiate 
the polynomial, and compnutes the error of the resulting derivative from the true 
value. Let's run it with x defined appropriately according to the given set of data 
points and see the results. 


>>nm540 
dfx( 0.78540) 
dfx( 0.78540) 
dfx( 0.78540) 


0.689072 (error: -0.018035) %with Xx = [1:3]*pi/8 
0.706556 (error: -0.000550) %with X [0:4]*pi/8 
0.707072 (error: -0.000035) %with X [2:6]*pIi/16 


This illustrates that if we have more points that are distributed closer to the target 
point，we may get better Tesult. 


%nm540 
s% to interpolate by Lagrange polynomial and get the derivative 
Clear，CLf 
X0 = pi/4; 
df0 = cos(x0); 5 True Value of derivative of Sin(x) at x0 = pi/4 
form= 1:3 
If m == 1，X [1:3]*pi/8; 
elseif m== 2，X= [0:4]*pi/8; 
el1Se Xx= [2:6]*pi/16; 
end 
yY = Sin(x); 
px = Jagranp(x,y); Lagrange polynomial interpolating (xy) 
dpx = polyder(px); derivative of polynomial px 
dfx = polyval(dpx，Xx0) ; 
fprintf(” dfx(%6.4f) = 
end 


2 
-6 
2 

-6 


#%510.6f (error: 5%510.6f)\n' ，Xx0,dfx,dfx - df0); 





One more thing to mention before closing this section ls that we have the 
MATITLAB built-in routine“diff()”， which fnds us the difference vector for a 
given vector. When the data points {(xk， 太 xb) ,大 一 112,...)are given as an 
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ASCII data file named“xy.dat”，we can use the Toutine“diff()”to get the 
divided difference,，which is Similar to the derivative of a continuous function. 


>>1J10ad xy.dat s%input the contents of :xy.dat' as a matrix named Xy 
>>dydx = diff(xy(:,，2))./diff(xy(:,1)); dydx' %dqivided difference 
dydx = 2.0000 0.50000 2.0000 


7 1JCD XI 一 区 Je 一 Jo P Je 一 了 Co) 
大 xy(:,1) xy(:,2) diff(xy(:,1)) diff(xy(:,，2)) 由 肥 XK1 一 XK 
1 一 1 儿 ] 人 2 
2 0 4 和 1 1/2 
3 乡 5 一 ] 一 2 2 
4 1 3 


5.5 _ NUMERICAL INTEGRATION AND QUADRATURE 


The general form of numerical integration of a function 太 (x) over Some interval 
[ac, pb] is a weighted Sum of the function values at a finite number (W 十 ]) of 
Sample points (nodesj, referred to as “quadrature : 


六 N 
/ JoDdz 宕 》 wkjGo) with =xo<xl< :<XN 一 D (3.5.1) 


K 人 一 0 


Here, the sample points are equally spaced for the midpoint rule, the trapezoidal 
rule, and Simpson's rule, while they are chosen to be zeros of certain polynomials 
for Gaussian quadrature. 

Figure 3.3 Shows the integrations over two Segments by the midpoint rule， 
the trapezoidal rule, and Simpson's rule, which are referred to as Newton 一 Cotes 
formulas for being based on the approximate polynomial and are Implemented 
by the following formulas. 


XK 十 1 


(midpoint rule)》 JJ dx 宕 屎 万 天 (3.9.2) 
With 亚 XiHL 一 和 区， JE 三 oo ，Xmk 一 二 
区 十 1 万 
(trapezolidal rule) Jrx) dx 三 了 (大 十 AD (3.9.3) 
with 严 王 Xi 一 砍 ， 有 太 三 Jo 
XK 十 1 万 
(Simpson s rule) 1/ Jrx) dx 三 了 Wi-1 十 4 斥 十 大 (3.5.4) 


XK+1 一 XK 一 1 


with /一 
人 
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Xk-1 Xk Xk+1 MT Xk Xk+1 
(a) The midpoint rule (b) The trapezoidal rule 
= 上 
MT XK / Xpk+1 


(c) Simpson's rule 


Figure 5.3 Various methods of numerical integration. 


These three integration rules are based on approximating the target function 
(integrand) to the zeroth-, first- and Second-degree polynomial, respectively. Since 
the first two integrations are obvyious,， we are going to derive just Simpson's rule 
(9.3.4). For Simplicity，we Shift the graph of jx) by 一 允 along the x axis， 
or, equivalently, make the variable substitution 上 一 X 一 人 So that the abscissas 
of the three points on the curve of jx) change from x 王 { 一 ]X 了 十 用 
to 1 一 { 一 六, 0, 十 由 .Then, in order to find the coefficients of the Second-degree 
polynomial 

Po(D 一 cl 三 十 cf 十 c3 (5.5.5) 


matching the points (一 六 太 -1), (0, 太 ), (十 ， 帮 TD)，we Should solve the follow- 
ing Set of equations: 
P2( 一 如 = cl( 一 凡 7 十 cz( 一 站 十 cs = 太 - 
P2(0) = cl102 十 cz0 十 c3 = 办 
P2( 十 四 = cl( 二 1) 十 cz( 十 站 十 cs = 万 1 


to determine the _ coefficients cl, cz，and c3 as 


二 演员 一 帮 1 二 大 1 有 ji+ 大 LI 1 
人 大 ， 2 ? 刀 2 2 天 
Integrating the Second-degree polynomial (3.35.3) with these _ coefficients from 
1 一 一 ntot 一 彤 yields 
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严 


1 ] ; 1 六 2 3 
Da2(1)QL 一 一 cl 十 二 cz 弓 十 c3t 一 二 C17- 十 2c3 及 
3 2 3 





一 忆 


2 (jd 大 -1 
3 2 


万 
一 灰 十 5】 全 了 WE-1 十 4 大 十 帮 HD 


This is the Simpson integration formula (3.3.4). 
Now, as a preliminary work toward diagnosing the errors of the above inte- 
gration formulas,， we take the Taylor series expansion of the integral function 


2 | JOd with goo0 = fo0，g200 = Po0， geCD = Jon 


(3.5.0) 
about the lower bound xx of the integration interval to write 


1 1 
8GC0) 一 8C 十 8 CC 一 zt) 十 8 Co 人 一 XU) 十 了 GCCG 一 如 三 十 … 


Substituting Eq. (3.3.0) together with x = Xk+Hl and xk+1 一 允 一 六 intothis yields 
JXK+1 万 2 万 3 万 4 15 
jd =0+ 风 CD 二 Co 可 太 CD 二 页 太 CD 二 可 Go 十， 


(9.9.7) 
First, for the error analysis of the midpoint rule, we Substitute X_1 and 一 丑 一 
X_1 一 XUinplace of x+land7inthis equation to write 


AXK 


JIK-1 万 2 万 3 万 4 75 
]/ Ja =0 一 MGoD 二 万 六 Co 一 Se 十 页 太 om 一 2 十 …: 


and Subtract this equation from Eq. (3.3.7) to write 


XK 十 1 


foe-1/ fow= 1/ foex+ 上/ 而 起 
Tt1 213 O) 275 d) 
攻 1 jd =2if60+ 本 Jo00+ 本 JoDT 6G53) 


Substituting xx and XiK 一 (十 XI)/A2 in place of xl and xx in this equation 
and noting that Xk+l 一 Xi 一 Xi 一 XU 一 站 /2 We obtain 








XK+1 太 3 
Jo dx 三 有 Comk) 十 本 广 帮 OOComh) 
15 
(4) 
二 
XK+1 太 3 态 5 
Ja 一 寻 roD= 二 太 CobDTTo01 Co 二 一 Or) 


AK 


(5.5.9) 
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This，together with Eq. (9.3.2)，implies that the error of integration Over one 
Segment by the midpoint rule is proportional to /3. 

Second, for the error analysis of the trapezoidal rule，we Subtract Eq. (9.5.3) 
from Eq. (3.3.7) to write 


XK 二 1 万 
人 JCx) dx 一 了 了 (C/Cxo) 十 jck+l)) 

7 3 74 5 

=MfCo 十 三 帮 C 十 2 Ceb) 十 Ce) 十 SR Ge) 十 …， 

有 / 刀 (2) 及 (3) 
一 了 J Cr) 十 Fe) 十 天 广 (xxe) 十 人 (CCk) 十 4 (CD) 
1 
+ 布 太 CD 十 1 


网 ) 和 ye 和 7 OUo = O(0D) (3.5.10) 
二 让 Co 一 二 了 0 CTOJ) 三 OO ) .3. 


This implies that the error of integration over one Segment by the trapezoldal 
rule is proportional to 73. 

Third, for the error analysis of Simpson's rule，we Subtract the Taylor series 
expanslion of Eq. (3.3.4) 


万 
了 We-D) 十 48(xck) 十 FeHlD)) 
2 7o 2 raq 
= 了 Jr) 十 48xk) 十 Je) 十 本 了 (Ce) 十 本 (Cr) 十 …: 


刀 3 5 
一 272xb) 十 本 太 Co) 十 了 7 Ci 十 …， 


from Eq. (3.9.8) to write 


XK+1 六 有 5 
下 Jar- 了 CoxeDT41CD+yGerD)= 二 +Ow7) 


= 0(015) (5.5.11) 


This implies that the error of integration over two Segments by Simpson's rule 
is proportional to 5. 

Before closing this section，letf's make use of these error equations to find 
a way of estimating the error of the numerical integral from the true integral 
without knowing the derivatives of the target (integrand) function (xzx)， For 
this purpose，we investigate how the error of numerical integration by Simp- 
Son' S rule 


万 
75 XeH 7) 一 本 避 0 十 48(xck) 十 大 CID) 
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will change iftthe Segment width is halved to 7/2. Noting that, from Eq. (5.5.11)， 


XK+1 75 
sp) 一 Je dx 一 人 CT XeHD 有 ) 祥 四 < [x_i xD) 


万 AH 万 
&s(5) 一 / JJ)dxz 一 5 (wa 2 


JXK 万 XK 十 1 
一 / 三) CX 和 75 (s， XK) 2 十 三) CQX 


大 


万 
一 (= XK+1， 3 (Ce [xz XI]) 


刀 /2 
久 一 22 Jo = 二 放 Es(O0) 


we can express the change of the error caused by halving the Segment width 


as 
万 
区 一 达 s (| 





万 
1 Xe P) 一 75 (aaa 间 ] 


2 


也 吉首 可 吉 移 | 训 [ (5.5.12) 
| SU)| 祥 57 ,9: 


This suggests the error estimate of numerical integration by Simpson' s rule as 


| 
SR 24 


Also for the trapezoidal rule, Similar result can be derived: 


本 |R 和 表征 
7\2 咱 ”到 2 二 1 


5.6 TRAPEZOIDAL METHOD AND SIMPSON METHOD 





万 
15(Cc 1 XkH 7) 一 75 (sa ] (5.5.13) 





万 
1T(Ce TD XH 万) 一 77 (sa 2 (5.5.14) 


In order to get the formulas for numerical integration of a function 太 (x) over 
Some interval [a, pg]，we divide the interval into NW Segments of equal length 
] 一 (一 0)/N Sothat the nodes (Sample points) can be expressed as {X 一 Q 十 
K1 ,一 0,1,2,...,N. Then we have the numerical integration of 太 (x) over 
[wa, bp] by the trapezolidal rule (3.3.3) as 


愉 K 十 1 


JJdr= > 人 yo)dr 


万 
研 本 (yo 十 万 ) 十 ( 户 十 万 ) 十 十 (A++ 太 -D)+(CA + 放 )} 
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内 Al 
172(a,D,7) 一 户 了 十 >》， 太 Cxco) (3$.6.1) 
ii 


whoseerroris proportionalto p2 as Ntimes the errorforone segment [Eq. (5.5.10)]， 
that 1s， 
NO(U3) = 一 dxO(03) = 00 


On the other hand，we have the numerical integration of 太 (x) over [a,p] by 
Simpson's rule (3.3.4) with an even number of segments N as 


N/2 一 1 22m 12 
上 Jo) dx 一 二 人 OoD dx 
太一 0 “32m 
户 
半 3{(yo 十 4 万 十 户 ) 十 ( 户 +4 户 十 万) 十 十 (二 4 + 放 )} 
N/2 一 1 N/2 一 1 
1sa(a, 2, 门 一 和 ro+y7ads 六 二 和 reoj| (5.6.2) 
1 一 0 11 一 1 


N/2-1 
-外 ra +7OD 二 2 》 ”Jon4D) 十 沪 ro 


1 一 0 大 一 1 


whoseerroris proportionalto 1 as N times the errorfor one segment [Eq. (5.5.11)]， 
that ls， 
(NO(05) = 一 dxO0(02) = OO 


These two integration formulas by the trapezoidal rule and Simpson's rule are 
cast into the MAILAB routines“trpzds()”and“smpsns ()”, respectively. 


function INTf = trpzds(f,ab;N) 

s%integral of f(x) over [a,b] by trapezoidal rule With N Segments 

if abs(b - al) < eps |N<= 0，INTf = 0; return;i end 

h= (bb-a)/Nix=a+[0:N]*h; fx = feval(f,Xx); values of ffor all nodes 
INTf = hx*((fx(1) + fx(N + 1))/2 + Sum(fx(2:N))); %Eq.(5.6.1) 


function INTf = Smpsns(f,ab,N,varargin) 

s%integral of f(x) over [ab]l by Simpson's rule with N Segments 

if nargin < 4，N = 100; end 

if abs(b - a)<1e-12 |N<= 0，INTf = 0; return;i end 

if mod(N,2) -=0,N=N+ 1; end s%make N even 

h= (bb-a)/ 人 Nix=ar+[o:NI*h; s%the boundary nodes for N segments 

fx = fevel(f,x,varargin{:}); %values of f for all nodes 

fx(find(fx == inf)) = realmaxji fx(find(fx == -Inf)) = -realmax; 

kodd 2:2:Nj keven = 3:2:N - 1; %the set of odd/even indices 

INTf h/3*(fx(1) + fx(N + 1)+4*Sum(fx(kodd)) + 2*Sum(fx(keven)));%Eq.(5.6.2) 
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5.7 RECURSIVE RULE AND ROMBERG INTEGRATION 


In this section，we are going to look for a recursive formula which enables us 
to use Some numerical integration with the Segment width 六 to produce another 
(hopefully better) numerical integration with half the segment width (7/2). Addi- 
tionally，we use Richardson extrapolation (Section 5$.1) together with the two 
Successive numerical integrations to make a Rombersg table that can be used to 
Improve the accuracy of the numerical integral Step by step, 

Let's Start with halving the Segment width to A/2 for the trapezolidal method. 
Then, the numerical integration formula (5.0.1) can be written in the recursive 
form as 


AN _ aoO+Fo 去 
1772 ( 必 3) 二 人 十 记 J 太 (xzkj2) 


忆 + 去 X1 
一 了 | 二 2 J(Cxc2mj2) 十 2 7GCenr) 


N-1 
] 
一 林 [ee D,7) 十 温 J (xzom+D/2)(terms for inserted mao| 
1 一 0 

(3.7.1) 
Noting that the error of this formula is proportional to 2(O(p2))，we apply a 
Richardson extrapolation [Eq. (3.1.10)] to write a higher-level integration formula 
having an error of O(Pp4) as 


22772(a， 2， 刀 ) 172(Q， D， 21) 


174(a, D,7) 一 2 
5 3 工 Sn 
JO+AODT2》 oo) 
并 二 二 
N/2-1 
ea 学 reo 
11 一 1 
六 N1/2 N/2 一 1 
-orvo+r 人 和 Jo _D 十 2 re 
(5.6.2) 


三 ”Tv(a,D 月 (5.7.2) 


which coincides with the Simpson's integration formula. IThis implies that we 
don't have to distinguish the trapezoidal rule from Simpson' s rule. Anyway， 
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TIeplacing 岂 by A/2 in this edquation yields 


, 已, 一 
22 二 1 


刀 22772(a,D, 7/2) 一 TIT2z(a,D, 门 
774 亲 人 


which can be generalized to the following formula: 


2 各 厅 (Q， 2， 2 一 《kt 站 es 厅 入 (Q， 2， 2 飞 门 
22 一 1] 
for 呈 二 1 KK 之 0 (3.7.3) 


厅 ra2o+D(a,D 2 -40D7) 一 


Now, it ls time to introduce a Systematic way, called Romberg integration, of 
Improving the accuracy of the integral Step by step and estimating the (trun- 
cation) error at each step to determine when to stop. It is Implemented by 
a Romberg Table (Iable 5.4)，that 1$，a lower-triangular matrix that we con- 
struct one IOow per iteration by applying Eq. (9.7.1) in halving the Segment width 
to get the nextrrow_ element (downward in the first column)，and applying 
Eq. (5.7.3) in upgrading the order of error to get the next-column elements 
(Tightward in the row) based on the up-left (north-westb) one and the left 
(wesb) one. At each iteration KK，we Use Eq. (9.5.14) to estimate the truncation 
erOT as 


瑟 2- 有 | 六 一 工 | 瑟 2C= 一 到 站 (24 5.7.4 
[Er ztD( )| | | 妆 和 人 ) 一 ZK 人 )| (5.7.4) 


and stop the iteration when the estimated error becomes less than Some prescribed 
tolerance. Then, the last diagonal element is taken to be “Supposedly” the best 


function [x,RerrN] = rmbrg(f,ayb,tol,K) 
%construct Romberg table to find definite integral of f over [ab] 
h=b-aiN=1; 
if nargin < 5，K= 10;j end 
R(1,1) = h/2*(feval(f,a)+ feval(f,b)); 
for k = 2:K 
h = h/2; N= Nx2ij 
R(k,1) = R(k - 1,1)/2 + hxsum(feval(f,a +[1:2:N - 1]*h)); %Eq.(5.7.1) 
tmp = 1; 
for n = 2:K 
tmp = tmp*x4; 
R(k,n) = (tmpxR(kin - 1)-R(kK - 1In- 1))/(tmp - 1); %Eq.(5.7.3) 
end 
err = abs(R(k,k- 1)- RIkK - 1;,k - 1))/(tmp - 1); %Eq,(5.7.4) 
if err < to1l， breakj end 
end 
x=Rk,k); 
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estimate of the integral. This sequential procedure of Rombersg integration ls cast 
into the MAITLAB routine“rmbrg () 

Before closing thlis section，we test and compare the trapezoidal method 
(trpzds()”，Simpson method (smpsns() 六 ，and Romberg _ integration 
Crmbrg()”) py trying them on the following integral 


Table 5.4 Romberg Table 





Iteration 大 Segment Width 天 瑚 一 1] 瑚 一 2 和 
0 Po0 17.2(po0) 
(5.7.0D 了 (05.7.3) 
1 2 -po 厅 2(2- po0) 一 >174(2 -70) 
(5.71) 了 (3.7.3) (5.7.3) 
2 2-/0 厅 20C 一 po0) 一 全 厅 4(022p0) 一 全 厅 6(02-p0) 


4 4 4 
人 40x0-oexdz=10|-2c2x0-ok+ zc-294z| 
0 0 





4 4 4 
-100|-2exq | e-2x(1 2 让 cd 
0 0 站 


4 
一 200x2e- 代 二 3200e-8 = 1.07348040929 (5.7.3) 





Here are the MAILAB statements for this job listed together with the run- 
ning results. 


>>f = linline(' 400*X.x(1 - X).xexp(-2*Xx) ，X' ); 

>>a= 0jb= 4;N= 80; 

>>format Short e 

>>true_I = 3200*exp(-8) 

>>It = trpzds(f,ab,N)，errt = It-true_I %trapezoidal 
It = 9.9071e-001，errt = -8.2775e-002 


>>IS = Smpsns(f,aib,N)，errs = IsS-true_I %Simpson 
INTfS = 1.0731e+000，error = -3.3223e-004 
>>[IR,R,err,N1] = rmbrg(f,ab,.0005)，errR = IR - true_I %5Romberg 


INTfr = 1.0734e+000，N1 = 32 
error = -3.4943e-005 


As expected from the fact that the errors of numerical integration by the trape- 
Zzoidal method and Simpson method are O(n2) and O(p)，respectively，the 
Simpson method presents better results (with Smajller error) than the trapezolidal 
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one with the Same number of Segments N 王 80. Moreover, Rombersg integration 
with N = 32 Shows a better Tesult than both of them. 


5.8 ADAPTIVE QUADRATURE 


The numerical integration methods in the previous sections divide the inte- 
gration interval uniformly into the Segments of equal width, making the error 
nonuniform over the interval 一 that 1Ss，SmalJlarge for smooth/swaying portion 
of the curve of integrand 太 (x). In contrast, the strategy of the adaptive quadra- 
ture ls to divide the integration interval nonuniformly into Segments of (gener- 
ally) unequal lengths 一 that 1$， ShortUlong segments for Swaying/smooth portion 
of the curve of integrand 太 x)，aiming at having smaller error with fewer 
Segments. 

The algorithm of adaptive quadrature Scheme starts with a numerical integral 
(INTf) for the whole interval and the sum of numerical integrals (INTf12 = 
INTf1 + INTf2) for the two segments of equal width. Based on the difference 
between the two SuccesSlve estimates INTf and INTf12, it esttmates the error of 
INTf12 by using Eq. (3.3.13)/(9.5.14) depending on the basic integration rule. 
Then, it the error estimate 18 within a given tolerance (tolj, it terminates with 
INTf12. Otherwise, it digs into each Segment by repeating the Same procedure 
with half of the tolerance (to1/2) assigned to both segments, until the deepest 
level satisfies the error condition. This is how the adaptive Scheme forms Sections 
of nonuniform width, as ilustrated in Fig. 39.4. In fact, this algorithm really fits 
the nested (recursive) calling Structure introduced in Section 1.3 and ls cast into 
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Figure 5.4 The subintervals (segments) and their boundary points (nodes) determined by the 
adaptive Simpson method. 
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the routine“adap_smpsn()”, which needs the calling routine“adapt_smpsn()” 
for start-up. 


function [INTf,nodes,err] = adap_smpsn(f,ab,INTf,tol,varargin) 

s#%5adaptive recursive Simpson method 

cC = (at+b)/2; 

INTf1 = Smpsns(f,ay cl1varargin{f:}); 

INTf2 = Smpsns(f,cb,1,varargin{f:}); 

INTf12 = INTf1 + INTf2; 

err = abs(INTf12 - INTf)/115; 55 Error estimate by Eq.(5.5.13) 

if isnan(err) | err < tol | tol<eps 5 NaN? Satisfying error? Too deep level1? 
INTf = INTf12; 
points = [acbl]; 

el1Se 

[INTf1,nodes1,err1] 
[INTf2,nodes2,err2] 
INTf = INTf1 + INTf2; 
nodes = [nodes1 nodes2(2:1length(nodes2) )]; 
err = err1 + err2; 

end 


adap_smpsn(f,a,c,INTf1,tol/2,varargin{:}); 
adap_smpsn(f,cb,INTf2,tol/2,varargin{:}); 


function [INTf,nodes,err] = adapt_smpsn(f,ab,tol,varargin) 
s#%apply adaptive recursive Simpson method 

INTf = Smpsns(f,ab,1,varargin{f:}); 

[INTf,nodes,err] = adap_smpsn(f,a,b,INTf,tol,varargin{:}); 





We can apply these routines to get the approximate value of integration 
(3.7.3) by putting the following MATILAB statements into the MAILAB com- 
mand window. 


>>f = inline(` 400*Xx.x(1 - X).xexp(-2*Xx) ，X'); 

>>a=0; b = 4; tol = 0.001; 

>>format Short e 

>>true_I = 3200*exp(-8) 

>>Ias = adapt_smpsn(f,aib,tol)，erras=Ias-true_I 
Ias = 1.0735e+000，erras = -8.9983e-006 


Figure 5.4 shows the curve of the integrand F(Cx) = 400x(1 一 x)e-2 together 
with the 23 nodes determined by the routine“adapt_smpsn()”，which yields 
better results (having Smaller error) with fewer segments than other methods 
discussed So far. From this fgure，we see that the nodes are dense/Sparse in the 
Swaying/smooth portion of the curve of the integrand. 

Here, we introduce the MAILAB built-in routines adopting the adaptive Tecur- 
Sive integration Scheme together with the illustrative example of their usage. 


"quad(f,ayb,toltrace;p1,p2,..)”/ "quadl(f,ab,tolitrace,p1;,p2，..)” 
>>Iq = quad(f,ab,tol)，errq = Iq - true I 
Iq = 1.0735e+000，errq = 4.0107e-005 


>>IqlLl = quadl(f,aib,tol)，errqLl = Iq1L1 - true I 
IqlL = 1.0735e+000，errq1 = -1.2168e-008 
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(cf) These routines are capable of passing the parameters (p1,p2,..) to the integrand 
(targeb function and can be asked to Show a list of intermediate Subintervals with 
the fifth input argument trace=1. 

(cf) quadl() is introduced in MAILAB 6.x version to replace another adaptive integration 
routine quad8() which is available in MATLAB 5.x version 


Additionally，note that MAILAB has a Symbolic integration IToutine 
“int(f,ab)”. Readers may type “help int”into the MAILAB command win- 
dow to see its usage，which is restated below. 


int(f) gives the indefinite integral of f with respect to its Independent 
Variable (closest to“x7). 

int(f,v) gives the indefinite integral of f(v) with respect to v given as 
the second input argument. 

int(f,ajb) gives the definite integral of f over [ab] with respect to jlts 
independent variable. 


int(f,v,aib) gives the definite integral of f(v) with respect to Vv OverT 
[a,b]. 


(cf) The target function f must be a legitimate expression given directly as the first 
input argument and the upperlower bound a,b of the integration interval can be 
a Symbolic scalar or a numeric. 





了 上 xample $.2. Numerical/Symbolic Integration using quad() /quadl1()/int()， 
Consider how to make use of MAILAB for obtaining the continuous-time 
Fourier series (CtFS) coefficient 


P/2 忆 /2 
X/ 一 1/ X(1e-yeoot dt 一 (er di (E5.2.1) 
一 P/2 一 P/2 


For Simplicity,， let's try to get just the 16th CtFS coeffticient of a rectangular 


Wave 
1] for 一 1<ft<1 


0 for -2<f<1lorl<fr<2 (5.2.2) 


X(1) 一 | 


which ls periodic in ! with period 已 = 4. We can compnute it analytically as 


2 1 1 
X16 = | xDe-72160 dr 一 / e-78r1d1 一 这 本 e-78m 

二 人 生 下 一 /8 1 
1 


=0 (E5.2.3) 
一 1 





1 
三 责 Sin(8 立 帮 
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%nm5e02 
%use quad()/dquad8() and int() to get CtFS _ coefficient X16 in EX 5.2 
ftn = "exp(-j*kxwOx*t)'; fcos = inline(ftn，t'，Kk' ，woO' ) 
P=4;Kk=16; wo = 2xpi/P; 

-1; b= 1; tol=0.001; trace = 0 
X16 quad = quad(fcos,ayb,toltraceyk,wo) 


X16_ quad1l1 = quad1l(fcosaybtoltrace kwoO) 

Syms ti; 5 declare Symbolic variab1le 

Iexp = int(exp(-j*kxwOx*t),t) 务 Symbolic indefinite integral 
Icos = int(cos(k*wOxt),t) 各 Symbolic indefinite integral 

X16 sym = int(cos(kxwoOxt)t,-1,1) 55 Symbolic definite integral 





As a_numerical approach，we can use the MAILAB routine “quad () 2/ 
“quad1()” On the other hand，we can also use the MATLAB routine“int()”， 
which is a Symbolic approach. We put all the statements together to make the 
MAILAB program “nm5e02”，in which the ffth input argument (trace) of 
“quad()2/ quadl1()”is set to 1 so that we can see their nodes and tell how 
different they are. Let's run it and see the results. 


>>nm5e02 
X16_ quad = 0.8150 + 0.0000i %betrayal of MATLAB? 
X16_quadl = 7.4771e-008 %5almost zero，0OKI! 
Iexp = 1/8*i/pix*exp(-8*i*xpixt) 5%(E5.2.3) by Symbolic computation 
Icos = 1/8/pix*sin(8*pixt) %5(E5.2.3) by Symbolic computation 
X16_ sym = 0 %exact answer by Symbolic computation 


What a Surprise!l It ls totally unexpected that the MAILAB routine “quad ()” 
gives us a qujte eccentric value (0.8130), even without any warning message. The 
routine“quad()”must be branded as a betrayer for a piecewise-linear function 
multiplied by a periodic function. This seemas to imply that “quad1()”is better 
than“dquad()”and that“int()”is the best of the three commands. It should， 
however, be noted that “int()”can directly accept and handle only the functions 
composed of basic mathematical functions, rejecting the functions deftined in the 
form of string or by the“inline()”command or through an m-fle and besides， 
it takes a long time to execute. 


(cf) What about our lovely routine“adapt_smpsn()”2> Regrettably，you had better not 
count on it, Since it will give the wrong answer for this problem. Actually，“quad1()” 
is much more relliable than“quad()”and“adapt_smpsn()”. 


5.9 GAUSS QUADRATURE 


In this Section，we cover Several kinds of Gauss quadrature methods 一 that ils， 
GausS-Legendre integration,， Gauss-Hermite integration，GausS-Laguerre inte- 
gration and Gauss-Chebyshev [LI integration. Each tries to approximate one of 
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the following integrations, respectively: 


忆 十 co 3 十 co 
1 (0D) d1， / e ”0) dt， 下 edt， 
QG 一 0 


Oo 


AN 
1 元 /ou 太 Maroa 2 wy 人 


The problem is how to fx the weight wi"s and the (Gauss) grid points 大?S 


5.9.1 Gauss-Legendre Integration 
HI the integrand JU) is a polynomial of degree < 3(=2N 一 1, then its inte- 
gration 
+1 
7 (一 1 1 一 J 三 (1) dt (3.9.1) 


can exactly be obtained from just 2(W) points by using the following formula 


ZI, 记 ] = VIJ GD 十 wz) (5.9. 轨 


How marvelous it ls! It is almost a maglic. Do you doubt it? Then, let's find the 
weights w1，Ww2 and the grid points 娘 , 已 Such that the approximating formula 
(9.9.2) equals the integration (9.9.1) for 7j 太 (0) = 1(of degree 0)，!(of degree ]1)， 
太 (of degree 2)，and 13(of degree 3). In order to do so，we should solve the 
following System of equations: 


JFOD=1: wifenD+wzfe)=wi+wo = 1dr 一 2 (5.9.3a) 
JJ 一 工 : w+ =wa+we= 人 ua (5$.9.3b) 
FOOD 一 2: WO 二 wa7p) = li 红 十 wo 节 三 刀 dt 一 3 (3.9.3c) 
FOOD = 已: wlFGD 十 wa 2p) = li 十 wz 她 三 已 di 一 0 (5.9.3d) 


Multiplying (5.9.3b) by 好 and Subtracting the result from (3.9.3d) yields 


U2( 太 一 在 ) = 2 放 ( 力 二 二) 人 一 向 =0 一 万 = 一 二 访 一 二 (meaningless) 
访 二 一 有 一 (5.9.3b)， (wl 一 wz) 廊 = 0， 


WU1 一 WU? 一 (95.9.3a)， 1 十 U1 一 2 


2 2 2 
WwW1 一 三 1 一 (5.9.3c)， 三 十 (一 站 三 了 厂 一 一 轧 三 一 


引 - 
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So that Eq. (9.9.2) becomes 


ZL 如 一 了 (- 扇 ) 十 上 (二 (5.9.4) 


We can expect this approximating formula to give us the exact value of the 
integral (9.9.1) when the integrand j 太 (D) ls a polynomial of degree < 3. 
Now,youare concernedabouthowto generallizethis two-pointGauss-Legendre 
integration formula to an NMW-point case, Since a System of nonlinear equation like 
Eq. (3.9.3) can be very difficult to Solve as the dimension increases. But,， don't 
worry about it. The N grid points ( 必 "S) of Gauss-Legendre integration formula 


N 
7crz[, 记 ,iv] 三 》， WNi (GD) (5.9.3) 
ij 


giving us the exact integral of an integrand polynomilal of degree < (CCN 一 ]) 
can be obtained as the zeros of the Nth-degree Legendre polynomial [天 -1， 
Section 4.3] 


LN/2 


ZNOD = 》 (一 


1 一 0 


三 :从 7 
人 (5.9.6a) 
2XilN -DIV 一 2 


] 
ZN(D) 三 站 (2N 一 TDIZN-1G) 一 (V 一 DZN-2(D) (5.9.0b) 


Given the N grid point 太 "$S，we can get the corresponding weight wNi s of the 
N-point Gauss-Legendre integration formula by Solving the System of linear 
equations 


1 1 1 。 1 UN 1 2 

妖 思 2 加 1N UN,2 0 
四 wow =| GCCDO/m (5.9.7) 
达 人 Lwww (一 (全 D>)/N 


where the th element of the Tight-hand Slide (RHS) vector ls 


1 1 1 
RHsow= 上 jdt 一 二 如 
有 


1 


1] 一 (一 ])" 


本 | 7 


(3.9.8) 





This procedure of fnding the N grid point 四 s and the weight wNi rs of the 
N-point Gauss-Legendre integration formula is cast into the MAILAB routine 
“Gauss1p()”. We can get the two grid point 汪 "S and the weight WwWNi s of the two- 
point Gauss-Legendre integration formula by just putting the following statement 
into the MAILAB command window. 
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function [tw] = Gausslp(N) 
ifN<0，fprintf('\nGauss-Legendre polynomial of negative order??\n ' ) ; 
el1Se 
t = roots(Lgndrp(N) )  ; %make it a row Vector 
A(1,:) = ones(1,N); bl(1) = 2; 
forn= 2:N5Eq.(5.9.7) 
Aln,:) =An - 1，:).x*t; 
if mod(n,2) == 0，b(n) 
else bln) = 2/nj;i 和 5 Eq.( 


党 [有 
5.9.8) 


- 1)ittLn-1tt)-(n- 1)Ln-2(t) Eq.(5.9.6b) 
= [1 0]! 
- 1)*[Lgndrp(N - 1) 0]-(N - 1)*[0 0 Lgndrp(N - 2)])7/N; 


function I = Gauss_Legendre(f,ab;N,Vvarargin) 


Gauss_Legendre integration of f over [ab] with N grid points 
Never try N larger than 25 
] = Gausslp(N) 
((b - a)x*t+ ar+b)/2; ss5Eq.(5.9.9) 
Xx = feval(f,x,varargin{f:}); 
= Wxfx'*(b - a)/2; %Eq.(5.9.10) 





>>[t,wW] = Gausslp(2) 
七 = 0.5774 -0.5774 W = 1 1 


Even though we are happy with the N-point Gauss-Legendre integration 
formula (3.9.1) giving the exact integral of polynomials of degree < (2N 一 了)， 
we do not feel comfortable with the fxed integration interval [一 1, 十 IJ]. But， 
we can be relieved from the Stress because any arbitrary finite interval [a, D] 
can be transformed into [一 1 十 1H] by the variable Substitution known as the 
GausS-Legendre translation 


六 一 ai 5 
和 二 2 出 (5.9.9) 





Then，we can write the NM-point Gauss-Legendre integration formula for the 
integration interval [a, p] as 


D Da 1 
1[a,D 本 FoDdz= 了 人 FeO)di 


(一 0) 庆 十 CQ 十 D 


NV 
厅 熏 
1 mxz= 一 wwrjoo) wihz= 5 


记 1 
(5.9.10) 
The Scheme of integrating jx) over the interval [a, p] by the NW-point GausS 一 
Legendre formula is cast into the MAILAB routine“Gauss_Legendre()” We 
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can get the integral (3.7.3) by Simply putting the following statements into the 
MAILAB command window. The result shows that the 10-point Gauss-Legendre 
formula yields better accuracy (Smaller error)，even with fewer nodes/segments 
than other methods discussed So far. 
>>f = inline('400*Xx.*(1 - X).*exp(-2*Xx)"，X'); %5Eq.(5.7.5) 
>>format Short e 
>>true_ I = 3200*exp(-8); 
>>a= 0jb= 4;N= 10; s%integration interval & number of nodes(grid points) 
>>IGL = gauss_1legendre(f,ab;N)，errGL = IGL-true _I 

IGL = 1.0735e+000，errGL = 1.6289e-009 


5.9.2 Gauss-Hermite Integration 


The Gauss-Hermite integration formula is expressed by Eq. (3.9.5) as 


N 
1ca[mp tv] = >》 wwiAG) (5.9.11) 
i=1 
and is Supposed to give us the exact integral of the exponential e 一 : multiplied 
by a polynomial 太 (1) of degree < (CCN 一 1) over (一 co, 十 co) 
+co  ， 
了 = |/ e 一 F0D)di (5.9.12) 
一 Co 


The N grid point 让 "S can be obtained as the zeros of the NW-point Hermite 
polynomial [ 玉 -1, Section 4.8] 


LN/ 刘 


厂 GD) 一 六 一 站 .CCV 一 2 二 DCDX (5.9.13a) 
记 0 7 . 
已 NG) = 21 瑟 GD) 一 克 (O) (5.9.13b) 


function [t,w] = Gausshp(N) 
ifN<0 
error('Gauss-Hermite polynomial of negative degree?? ) j 
end 
tt = roots(Hermitp(N)) ; 
Al(1,:) = ones(1;N); bl(1) sqrt(pi); 
forn= 2:N 
Aln,:) = An - :).xt;i %5Eq.(5.9.7) 
if mod(n,2) - 2)/2*b(n - 2); #%Eq.(5.9.14) 
el1Se 
end 
end 


function p = Hermitp(N) 
%Hn + 1(x) 2xHn(x)-Hn'(x) from 'Advanced Engineering Math' by Kreyszig 
if N <= 0， 
el1Se 
:N，p = 2*[p0]-[00polyder(p)]; end %5Eq.(5.9.13b) 
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Given the N grid point 太 "$S，we can get the welight wNi S of the NW-point 
GaussS-Hermite integration formula by solving the System of linear equations 
like Eq. (9.9.7), but with the Tight-hand side (RHS) vector as 





Co 
e-7 dy 


RHS(]) = |/ e 一 di 一 人 e 一 号 we/ 
一 / / ecoc+7) dx dy 一 / e- 六 27r 7d7 
= ,/ -re- 三 慰 (5.9.14al 


RHSCy) = 三 


一 Co 





Co 
1 
efn-1d1 一 人 (21e 一 一 mr2dr (=0ifnisevem 
人 一 2 


1 _12 112 co 1 _12 .13 1 
二 2 天 上- -2 人 1 epr3d -wm_-2RHSw 2 
2 了 2 


(3.9.14b) 

The procedure for finding the N grid point 太 "S and the corresponding weight 

WwWNMi s of the NMW-point Gauss-Hermite integration formula is cast into the MAT- 

LAB routine“Gausshp ()” Note that, even though the integrand function (g() 

doesn't have eas a multiplying factor, we can multiply it by eer = 1 to 
fabricate lt as if it were like in Eq. (3.9.12): 


7 = 记 8g(D) dr 一 三 ce (erg())dt = 三 ce 一 FOODdi (5.9.15) 


Oo 人 


5.9.3 Gauss-Laguerre Integration 


The Gauss-Laguerre integration formula is also expressed by Eq. (9.9.3) as 


N 
7cra[2,, 太 ] 三 》， WwWNji GD) (9.9.10) 
i=1 

and is Supposed to give us the exact integral of the exponential e multiplied 

by a polynomial 太 () of degree < (2N 一 1 over [0, co) 

Oo 
了 1 e 一 GD)di (5.9.17) 
0 


The N grid point 帮 "S can be obtained as the zeros of the Nth-degree Laguerre 
polynomial [ 玉 -1, Section 4.7] 





LT NT 
ZNCD) 一 > 了 二 (5.9.18) 
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Given the N grid point 万"$S，we can get the corresponding weight wNi s of the 
N-point Gauss-Laguerre integration formula by solving the System of linear 
equations like Eq. (3$.9.7), but with the rightrhand Slide (RHS) vector as 


Oo 


一 1 (3.9.19a) 


RHS(1) 一 人 e 一 df 一 一 ce 
6 0 





人 Oo Oo 
RHS(2) = 人 etH1 di1 一 -ee | 拉 (1 5 CI di 
4 0 


= 一 DJRHSw 一 了 (5.9.19b) 


5.9.4 Gauss-Chebyshev Integration 
The Gauss-Chebyshev I integration formula is also expressed by Eq. (3.9.5) as 


N 
ccl[0n ziv] 三 》， WwWNji GD) (9.9.20) 
证 


and is Supposed to give us the exact integral of 1/V1L 一 妃 multiplied by a 
polynomial 太 () of degree < (CCN 一 1 over [一 1 十 菇 


+1 
7 一 一 -一 
-1 V1 一 姑 


The N grid point 太 S are the zeros of the Nth-degree Chebyshev polynomial 
(Section 3.3) 


DOD dt (5.9.21) 


2 
人 (5.9.22) 
2 


and the corresponding weight wN.i S are uniformly Selected as 
WwWNi 三 和 区/N， VE (3.9.23) 


The Gauss-Chebyshev IJ integration formula is also expressed by Eq. (3.9.5) as 
N 

7ccz[0,2,, 太 ] 三 》， WwWNji (GD) (5.9.24) 
i=1 


and is supposed to give us the exact integral of V1 一 万 multiplied by a polyno- 
Imial 太 (0) of degree < (2N 一 1 over [一 1 十 匡 


+L! 
7 一 V1 一 刀 太 di (5.9.29) 
一 1 
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The N grid point 太 "S and the corresponding weight wNi Ss are 


7 芝 .”/( ， 
大 一 COS WN 1i 一 Sin for 一 12,...,，N 
Nd+l NT+l NT+l 
(3.9.20) 











5.10 DOUBLE INTEGRAL 


In this section，we consider the numerical integration of a function 三 (x, y) with 
ITespect to two variables x and y over the integration region 尺 一 {(x, y)la 和 YX 苹 
D,cC)<y<dx)} as depicted in Fig. 9.3. 


忆 do 
= 外 Je nero= | Jepoj|w ($.10.1) 
尺 CQ cr) 


The numerical formula for this double integration over a two-dimensional region 
takes the form 


14 和 
下 人 基 放 0) 三 广 二 》 本 太志 辣 (5.10.2) 
110 一 1 1 一 1 


where the weights wm, vn depend on the method of one-dimensional integration 
we choose. 



























户 x hx2 jxa3 二 二 证 汪 hx 六 
X Xp XM 


个 
人 
1 ao0 血 单 
意 | 由 
避 | 
| | ， 和 由 
1 | 下 
用 hx1， | 

hxo， y21 | 

hxo， 由 1 | 
| 区 
X0 


Figure 5.5 Aregion for a double integral. 
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(c) The MAILAB built-in routine dblquad() can accept the boundaries of integration 
ITegion only given as numbers. Therefore, 让 we want to use the routine in compnut- 
jing a double integral for a nonrectangular region 刀 , we Should define the integrand 
function 太 (x, y) for a rectangular region 尺 2 忆 (containing the actual integration 
Tegion 站 ) in Such a way that jx,y) 三 0for (xy) 和 Dithat is, the value of the 
function becomes zero outside the integration Tegion 刀 ,， which may result in more 
computations， 


function INTfxy = int2s(f,ab,c， 
s%5double integral of f(x,y) Over (xy)la <= X<= bc(x) <=y<= d(x)} 
%5 Using Simpson's _ rule 
if ceil(M) -= floor(M) %5fixed Width of segments on X 
hx= M;M=ceil(b - a)/hx)i 
end 


if mod(M,2) ~= 0 M+1; end 


了 天 遇 
hx = (b- a)/M; m 1:M+1; X= at+ (m- 1)*hx; 
由 了 s%if C is given as a constant number 
)); %in case C is given as a function of X 


了 
if isnumeric(Cc)，Ccx(m) = C 
else cx(m) = feval(c,x(m 
end 
if isnumeric(d)，dx(m) = di %if C is given as a constant number 
elSe dx(m) = feval(d,Xx(m)); %in case d is given as a function of X 
end 
if ceil(N) -~= floor(N) %5fixed Width of Segments on y 
hy = Ni; Nx(m) = ceil((dx(m)- cx(m))Vhy); 
ind = find(mod(Nx(m),，2) -= 0); Nx(ind) = NX(ind) + 1; 
else  %fixed number of Subintervals 
if mod(N,2) -==0，N=N+li end 
Nx(m) = N; 
end 
form= 1:M+1 
SXx(m) = Smpsns_fxy(f,Xx(m)，cx(m) ,dx(m)，Nx(m) ); 
end 
kodd = 2:2:M;i keven = 3:2:M - 1; %the set of odd/even indices 


INTfXxy = hx/3*(Sx(1) + SX(M + 1) + 4*sum(SX(kodd)) + 2x*Ssum(SX(keven))); 


function INTf = Smpsns_fxy(f，Xx，c，d，N) 

s#%51-dimensional integration of f(x,y) for RYy = {c <= y <= d} 

if nargin < 5，N = 100; end 

if abs(d - C)< eps |N<= 0，INTf = 0; return;i end 

if mod(N,2) -==0，N=N+1; end 

h= (dd- CI/NiVY= c+[0:N]*h;i fxy = feval(f,xy); 

fxy(find(fxy == inf)) = realmax;i fxy(find(fxy == -inf)) = -realmax; 

kodd = 2:2:Nj keven = 3:2:N - 1; %the Set of odd/even indices 

INTf h/3*(fxy(1) + fxy(N + 1) + 4*Sum(fxy(kodd)) + 2*sum(fxy(keven) ) ) ; 


s%nm510: the volume of a sphere 

xXx= [-1:0.05:1];，y= [0:0.05:1]; [X,Y] = meshgrid(x,y) 

f510 = inline('Sqrt(max(1 - X.xx - yxy0)) ，X'，yY') 

Z = Tf510(X,Y); mesh(x,y;,Z) ; 

a= -1ijib=1;c=0; d=inline(' sqrt(max(1 - X.xx;0)) ，Xx'); 

Vs1 = int2s(f510,ab,c,d,100,100) %with fixed number of Segments 
error1 = Vs1 - pi/3 

VsS2 = int2s(f510,ab,c,d,0.01,0.01) %with fixed Segment Width 
error2 = VS2 - pi/3 
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Figure 5.6 One-fourth (1/4) of a sphere with the radiusr = 1. 


Although the integration rules along the x axis and along the > axis do not need 
to be the same, we make a double integration routine“int2s(f,ajbycd,M;N)” 
which uses the Simpson method in common for both integrations and calls another 
routine“smpsns_fxy()”for one-dimensional integration along the y axis, The 
lefyright boundary ay/b of integration reglion given as the Second/third input argu- 
ment must be a number, while the lowerupper boundary c/d of integration region 
given as the fourth/fifth input argument may be either a number or a function 
of x. I the Sixth/seventh input argument M/N is given as a positive integer， 并 
will be accepted as the number of Segments; otherwise, it will be interpreted as 
the segment width PVA We also constructed a MAILAB program“nm510” 
in order to use the routine“int2s()”for fnding one-fourth of the volume of a 
Sphere with the radius 7 = 1 depicted in Fig. 3.0. 


1 A/ 1 一 X 
证) 中 VI- 妈 -ddxz= 了 = 1.04719755..， (5.10.3) 
一 1 J0 


JInterested readers are recommended to work with these routines and run the 
program “nm510.m” to see the result. 


>>nm510 
VS1 = 1.0470，error1 = -1.5315e-004 
VS2 = 1.0470，error2 = -1.9685e-004 
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PROBLEMS 


S.1 Numerical Differentiation of Baslic Functions 


H we want to find the derivative of a polynomialtrigonometric/exponential 
function， it would be more convenient and accurate to use an analytical 
computation (by hand) than to use a numerical computation (by computer). 
But，in order to test the accuracy of the numerical derivative formulas， 
Consider the three basic functions as 


六 Ge) = 一 2x， 户 () = sinx， 方 (Cr) 一 er (P5.1.1) 


(a) To find the first derivatives of these functions by using the formulas 
(9.1.8) and ($.1.9) listed in Table 3.3 (Section 9.3), modify the program 
“nm5p01.m”"，which uses the MAILAB routine“difapx()”(Section 
5.3) for generating the coefficients of the numerical derivative formulas. 
Fill in the following table with the error results obtained from running 
the program. 


用 二 大 一 户 十 8 万 一 8 广 1 十 三 2 
127 


一 1.00000000 
国 风 9.1038e-15 
一 0.30000000 


First Derivatives 一 一 一 一 一 


(e")| xz=0 而 汪 3.3373e-06 
一 1.00000000 
1.6667e-05 


%nm5p01 

f= inline( 

n= [1 -1]; 

cC= difapx(1;n); : ， 

num = Cx*feval(f,x0O + (人 1 ; drv = numy/h; 
fprintf(' with h = 5%6， drvdrv - DT); 





(b) Likewise in (a),， modify the program“nm5p01.m” in Such a way that 
the formulas (3.3.1) and (3.3.2) in Table .3 are generated and used to 
fnd the second numerical derivatives. Fill in the following table with 
the error results obtained from running the Program. 
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0 演 一 16 广 一 30 16 广 1 一 广 


CC3 一 22)O)| 2.6654e-14 


一 0. nn 


GinmOlnp 中 
一 一 0.86600254037 


一 1.0000000000 


S.2 Numerical Differentiation of a Function Given as a gsSet of Data Pairs 





Consider the three (numerical) functions each given as a set of fve data 
pairs in Table P5.2， 


Table P5.2 ”Three Functions Each Given as a Set of Five Data Pairs 


[am Tom | io To Tiom 


(a) Use the formulas (9.1.8) and (3.1.9) to find the first derivatives of the 
three numerical functions (at x 一 1, 1.0472 and 0, respectively) and fl 
in the following table with the results. Also use the formulas (9.3.1) 
and (9$.3.2) to find the second derivatives of the three functions (at 
X 一 1, 1.0472 and 0, respectively) and fll in the following table with 
the results. 


和 
First derivative by Eq. (5.1.8) “| 1.0000e-02 | 2.0000e-03 
First derivative by Eq, (5.1.9) = 2.5000e-04 国 国 国 





站 jb PoDlhaooz| Acoolco 
Second derivative by Eq. (5.3.1) 顺 玖 到 6.0254e-03 国 罗 
Second derivative by Eq. (5.3.2) | 2.4869e-14 国光 8.3333e-04 
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(b) Based on the Lagrange/Newton polynomial matching the three/five 
points around the target point, find the firstsecond derivatives of the 
three functions (at x 一 1，1.0472 and 0, respectively) and fill in the 
following table with the results， 


一 Tec 


First derivative on 1 (x) 人 | 1.0000e-03 6 
First derivative on 14) |4320le-12| |41667e-04 


各 立 2 
7 


Second derivative on /2(x)|1.4211e-14 加 0.0000e 十 00 
Second derivative on 14(X) | 0.8387e-03 [EE5 





S.3 First Derivative and Step-Size 


Conslider the routine“jacob()”in Section 4.6, which is used for computing 
the Jacobian 一 that is, the first derivative of a vector function with respect 
to a vector variable. 

(a) Which one is used for computing the Jacobian in the routine “jacob ()” 
among the first derivative formulas in Section 9.1? 

(b) Expecting that Smajller step-Size 凡 would yield a better solution to the 
Problem given in Example 4.3,， Bush changed h = 1e-4 to h = 1e-5 in 
the routine “newtons()”and then typed the following statement into the 
MAILAB command window. What solution could he get? 


>>rn1 = newtons('phys' ,1e6,1e-4,100) 


(c) What baffled him out of his expectation? Jessica diagnosed the trouble 
as caused by a Singular Jacobian matrix and modified the statement “dx 
= -jacob()\fx(:) ”intheroutine “newtons()”as follows. What solu- 
tion (to the problem in Example 4.3) do you get by using the modified 
routine, that is,，by typing the same statement as in (b)? 


>>prn2 = newtons( phys ,1e6,1e-4,100)，phys(rn2) 


J = jacob(f,xx(k,:),h,varargin{f:}); 
If Prank(J) < Nx 
- 1; 


fprintf('Jacobian Singular! det(J) = 5%512.6e\n ,det(J)); break ; 
el1Se 

dx = -JNfx(:); %-[dfdx] 人 -1x*fx; 
end 
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(dJ) To investigate how the accident of Jacobian Singularity happened, add 
h = 1e-5 to the (tentative) solution (rn2) obtained in (c). Does the 
result differ from rn27 开 not，why? (See Section 1.2.2 and Prob- 
lem 1.19.) 


>>rn2 + 1e-5 ~= rn2 


(e 


旋 


Charley thought that Jessica just circumvented the Jacobian singularity 
problem. To remove the source of singularity, he modified the formula 
(3.1.8) into 


Dr， 刀 ) 一 人 (P5.5.3) 


and Implemented it in another routine“jacob1()”as follows. 


function g = jacob1(f,xih,varargin) %Jacobian of f(Xx) 
if nargin<3，h =.0001; end 
h2 = 2*h;j N = length(x); 工 = eye(N); 
forn= 1:N 
if abs(x(n))<.0001，x(n) =.0001; end 
delta = hx*x(n); 


(n,:)*xdelta; 
feval(f,x + tmp,varargin{:}); 
feval(f,x - tmp,varargin{:}); 
(f1 - f2)/2/deltaji g(:;，n) = 





Withh=1le-5orh = 1e-6and jacob() replaced by jacob1() in 
the routine“newtons ()”, type the Same statement as in (c) to get a 
Solution to the problem in Example 4.3 together with its residual error 
and check if his scheme works fine. 


>>rn3 = newtons('phys ' ,1e6,1e-4;,100)，phys(rn3) 


S.4 Numerical Integration of Basic Functions 


Compnute the following integrals by using the trapezoidal rule, the Simp- 
Son's rule, and Romberg method and fll in the following table with the 
Tesulting errorS. 


/2 1 
人 ) 人 0o3-2zdr 人 sinxdx ( 动 ) 1 edx 
0 0 0 
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Trapezoidal | Simpson Romberg 
NV Rule Rule (tol = 0.0005) 
人 区 -2x)dr=0 国 吨 机 测 Ce 
DIE 


/2 1.2884e-2 四 风 风 8.4345e-6 
人 Sinxdx 一 1 
0 


D 且 [E 


edx = 0.63212055883 





S.S Adaptive Quadrature and Gaussian Quadrature for Improper Integral 


Consider the following two integrals. 


D 太 - d -2 =。 P5.5.1 
0 人 元 二 =2z0|- (P5.5.1) 
二 0 1 1 1 

(i /ec= 人 e+ 本 (P5.5.2) 


(a) Type the following statements into the MAILAB command window to 
use the integration Toutines for the above integral. What did you get? 
H Something ls wrong，what do you think caused it? 


>>f = inline('1./sqrt(x)"，Xx'); 5 define the integrand function 
>>Smpsns(f,0,1,100) % integral over [0,1] with 100 segments 
>>rmbrg(f,0,1,1e-4) 55 With error tolerance = 0.0001 
>>adapt_smpsn(f,0,1,1e-4) 点 With error tolerance = 0.0001 
>>gauss_1egendre(f,0,1,20) %5Gauss-Legendre with N = 20 grid points 
>>quad(f,0,1) 5 MATLAB built-in routine 

>>quad8(f,0,1) 各 MATLAB 5.X built-in routine 
>>adapt_smpsn(f,-1,1,1e-4) %integral over [-1,1] 

>>quad(f,-1,1) 各 MATLAB built-in routine 

>>quadl(f,-1,1) 5 MATLAB built-in routine 


(b) Itha decided to retry the routine “smpsns ()”, but with the Singular point 
excluded from the integration interval. In order to do that, she replaced 
the Singular point (0) which is the lower bound of the integration inter- 
val [0,1] by 10-4 or 10-5, and typed the following statements into the 
MATILAB command window， 


>>Smpsns(f,1e-4,1,100) 
>>Smpsns(f,1e-5,1,100) 
>>Smpsns(f,1e-5,1,1e4) 
>>Smpsns(f,1e-4,1,1e3) 
>>Smpsns(f,1e-4,1,1e4) 
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What are the results? Will it be better 寺 you make the lower-bound of 
the integration interval closer to zero (0), without increasing the number 
of segments or (equlivalently) decreasing the Segment width? How about 
increasing the number of segments without making the lower bound 
of the integration interval closer to the original lower-bound which ils 
Zero (0)? 

(c) For the purpose of improving the performance of “adap_smpsn() ”， 
Vania would put the following statements into both of the routines 
“smpsns()”and “adap_smpsn()”. Supplement the routines and check 
whether her idea works or not. 


EPS = 1e-12; fa = feval(f,avarargin{f:}); 
if isnan(fa)labs(fa) == inf，a = ar+max(abs(a)*EPS,EPS); end 


fb = feval(f,b,varargin{:}); 
?3 ?3??3?2??2?32?32?3?33? ?3 ?3?3??? ? ?? ? ? ?3?33?3?33?32?332?32?32?32?2? ?32? 





S.6 Various Numerical Integration Methods and Improper Integral 


Consider the following integrals. 








Co 100 ，， 
人 全 人 全 全 全 上 2 (P5.6.1) 
0 汪 2 0 
Re 2 1 
1 e dx 一 二 V 苑 (PS.60.2) 
0 凡 


Note that the true values of these integrals can be obtained by using the 
Symbolic computation command“int()”as below. 


>>Syms X，int(Sin(x)/x,0,inf) 
>>int(exp(-x^2),0,inf) 


(cf) Don't you believe it without Seeing it? Blessed are those who have not Seen 
and yet believe. 


(a) To apply the routines like“smpsns()”,“adapt_smpsn()”，“Gauss 
Legendre()”and“quadl1()”for evaluating the integral (PS.6.1)，do 
the following. 


(人 Note that the integration interval [0，co) can be changed into a 
finite interval as below. 


2 SinxX 1 sin x % Sinx 
如 起 二 axX 十 CQX 
0 区 0 区 1 区 
了 rs 1 1 
Sin X w+ sin(1/y) ( -二 沁 
0 大 1 1/y 了 


2 站 1 
= 上 2 w+ 和 局/ 蚊 沿 本 1 
0 出 


闷 0 


?9 
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(ii Add the block of statements in PS.3(c) into the Toutines“smp - 
sns()”and “adap_smpsn()”to make them cope with the cases 
of NaN (Not-a-Number) and Inf (Infinity). 


(ii Supplement the program “nm5p06a.m” So that the various routines 
are applied for computing the integrals (P3.6.1) and (P5.6.3), where 
the parameters like the number of segments (N = 200), the error 
tolerance (tol = le-4), and the number of grid points (MGL = 20) 
are Supposed to be used as they are in the program. Noting that 
the second integrand function in (P5.6.3) oscillates like crazy with 
higher frequency and larger amplitude as y gets closer to zero (0)， 
set the lower bound of the integration interval to a2 = 0.001. 


(ivy) Run the Supplemented program and fll in Table P3.6 with the 
absolute errors of the results. 


s%nm5p06a 
warning off MATLAB:divideByZzero 
fp56a = inline('Ssin(x)./X'，X'); fp56a2 = inline('sin(1./y)./y' ，y' ); 
IT = pi/2;j % True value of the integral 
a=0ijb=100;N=200; tol= 1e-4;MGL =20;jal=0ijbl=1;ia2=0.001; b2=1; 
format Short e 
es = Smpsns(fp56a,a,b;N)-IT 
e_as = adapt_smpsn(fp56a,a,b,tol)-IT 
e_ql = quadl(fp56a,ab,tol)-IT 
e_GL = Gauss_Legendre(fp56a,ab,MGL) -IT 
e_Ss = Smpsns(fp56a,al,b1,N) + Smpsns(fp56a2,a2,b2;,N) -IT 
e_Iasas = adapt_smpsn(fp56a,al,bl,tol)+ ... 
人 
- 工 
warning on MATLAB:divideByZzero 


%nm5p06b 
warning off MATLAB:divideByZzero 
fp56b = inline(' exp(-X.*Xx)，X'); 
fp56b1 = inline('ones(Size(x)) ，X' ); 
fp56b2 = inline('exp(-1./y./y)./y./y' ，y') 
a=0jb=200;N= 200;j tol = 1e-4; IT= Sqrt(pi)/2; 
al=0j bl= 1; a2=0j b2= 1; MGH = 2; 
es = Smpsns(fp56b,a,b;,N) -IT 
e_as = adapt_smpsn(fp56b,a,b,tol)-IT 

= quad(fp56b,a,b,tol)-IT 

= Gauss_Hermite(fp56b1,MGH)/2-IT 

Smpsns (fp56b,al,b1,N) + Smpsns(fp56b2,a2,b2,;N)-IT 

Iasas = adapt_smpsn(fp56b,al,b1,tol)+ ... 


warning off MATLAB:divideByZzero 
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Table P5.6 ” Results of Applying Various Numerical Integration Methods for 
Improper Integrals 


cs] so | oo 
ec| | ses| | oo| | ises 





3 66 3 66 


(b) TIoapplytheroutines like“smpsns()”，“adapt_smpsn() ”quad() and 
“Gauss_Hermite() ”forevaluatingtheintegral (P3.6.2),dothefollowing. 


(iD Note that the integration interval [0, co) can be changed into a 
finite interval as below. 


号 2 2 8 这 
站 e dx = 2 w+ 上/ e dx 
0 0 1 





1 1 e-L2 
= e w+ dy (P53.6.4) 
0 0 了 


(ii) Compose the incomplete routine“Gauss_Hermite”like “Gauss __ 
Legendre”, which performs the Gauss-Hermite integration intro- 
duced in Section 3.9.2. 

(ij 动 Supplement the program “nm5p06b.m” So that the various routines 
are applied for computing the integrals (P5.0.2) and (P5.0.4), where 
the parameters like the number of segments (WN = 200), the error 
tolerance (tol = le-4) and the number of grid points (MGH = 2) 
are Supposed to be used as they are in the program. Note that the 
integration interval ls not (一 co, co) like that of Eq. (3$.9.12)，but 
[0, co) and so you Should cut the result of "Gauss_Hermite()”by 
half to get the Tight answer for the integral (P5.0.2). 

(iv) Run the Supplemented program and fll in Table P53.6 with the 
absolute errors of the results. 


(c) Based on theresults listed in Table P3.6, answer the following questions: 


( Among the routines“smpsns()”, “adapt_smpsn()” “quad()”， 
and“Gauss ()”, choose the best two ones for (P5.6.1) and (P5.6.2)， 
Tespectively. 


?9 


(ii) The routine“Gauss-Legendre()”works (badly, perfectly) even 
with as many as 20 grid points for (P5.6.1)，while the routine 
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Gii) 


(iv) 


“Gauss_Hermite()”works (perfectly，badly) just with two grid 
points for (P3.0.2). It is because the integrand function of (PS5.6.1) 
ls (far from，just like) a polynomial，while (PS.6.2) matches 
Eq. (3.9.11) and the part of it excluding e-5 ”is dust like，far 
from) a polynomial. 


function I = Gauss_Hermite(f,N,varargin) 


ft = feval(f,t,varargin{f:}); 
WxTft ; 





Run the following program“nm5p06c,m” to see the Shapes of the 
integrand functions of (P3.6.1) and (P5.6.2) and the second inte- 
gral of (P5.0.3). You can zoom in/out the graphs by clicking the 
Tools/Zoom_in menu and then clicking any point on the graphs 
with the leftUyright mouse button in the MAILAB graphic win- 
dow. Which one is oscillating furiously? Which one ls oscillating 
moderately?” Which one is just changing abruptly? 


s%nm5p06c 

Cl1f 

fp56a = inline(' Sin(x)./X' ，Xx'); 
fp56a2 = inline(' Sin(1./y)./yY ，y， 
fp56b = inline(' exp(-X.*xXx) ，X'); 
x0 = [eps:2000]/20; x = [eps:100]/1100; 
Subplot(221)，plot(xo,fp56a(Xx0)) 
Subp1lot(223)，plot(xo,fp56b(Xx0)) 

Subplot(222)，y = 1logspace(-3,0,2000); 1oglog(y,abs(fp56a2(y) ) 
Subplot(224)，y = 1logspace(-6,-3,2000); 1oglog(y,abs(fp56a2(y) 


) ; 


) 
) ) 





The adaptive_ integration routines 1like “adapt_smpsn()” and 
“quad()”work (badly，fine) for (P35.6.1)，but (fine，badly) for 
(P5.0.2)，From this fact，we might conjecture that the adaptive 
integration Toutines may be (ineffective, effective) for the integrand 
functions which have many oscillations，while they may be 
(effective，ineffective) for the integrand functions which have 
abruptly changing slope. To support thlis conjecture，run the 
following program“nm5p06d”，which uses the“quad()”Troutine 
for the integrals 


忆 
定 -dx withD= 100,1000,10000...，  (P5.6.59) 
1 大 


:Sin(1/y) 
一 一 一 d with a = 0.001, 0.0001, 0.00001, ...(P5.6.5b) 
0 7 
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s%nm5p06d 
fp56a = inline(' Sin(x)./Xx' ，Xx' ); 
fp56a2 = inline('Ssin(1./y)./yY ，y ); 
Syms X 
IT2 = pi/2 - double(int(Sin(x)/x;,0,1)) strue value of the integral 
disp(' Change of upper limit of the integration interval ) 
a=1;b= [100 1e3 1e4 1e7]; tol = 1e-4; 
for 工 1:1ength(b) 
Iq2 = quad(fp56a,ab(Ii),tol); 
fprintf('With b = %512.4e，err _ Iq = %12.4e\n' ，b(i),Iq2-IT2); 
end 
disp(' Change of lower limit of the integration interval ) 
= [1e-3 1e-4 1e-5 1e-6 0]; b2 = 1; tol = 1e-4; 
for LIL = 1:5 
Iq2 = quad(fp56a2,a2(I),b2,tol); 
fprintf(' With a2=%512.4e，err_Iq=%512.4e\n ，a2(1I) ,Iq2-IT2) 
end 








Does the“quad()”routine work stably for (P3.0.3a) with the 
changing value of the upper-bound of the integration interval” 
Does it work stably for (P5.6.3b) with the changing value of the 
lower-bound of the integration intervaly Do the results Support or 
defy the conjecture” 

(cf) This problem warns us that it may be not good to use only one routine 
for a computational work and suggests us to use more than one method 
for cross check. 


S.7 Gauss-Hermite Integration Method 


Consider the following integral: 
Oo 
er cosxdx 一 0 (PS.7.1) 
0 


Select a Gauss quadrature Suitable for this integral and apply it with 
the number of grid points N = 4 as well as the routines “smpsns()” 
“adapt_smpsn()”“quad() ”and “quadl1()”to evaluate the integral. In 
order to compare the number of Hoating-point operations required to achieve 
almost the Same level of accuracy, set the number of segments for Simpson 
method to N = 王 700 and the error tolerance for all other routines to tol 一 
10-5. Fill in Table PS5.7 with the error results. 


Table P5.7 The Results of Applying Various Numerical Integration Methods 


Simpson adaptive quad quadl 
(N =700) | (tol = 10-5) | Gauss | (tol = 10-5) (tol = 10-5) 
(P5.7.1) 


| aops | 4930 5457 1484 11837 52590 (with quad8) 


(p5.8.1) 
| aops | 5024 7757 下 林 28369 75822 
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S.8 Gauss-Laguerre Integration Method 
(a) As in Section 9.9.1，sSection 9.9.2, and Problem 3.6(b)，compose the 


(b 


(cb) 


\ 必 


MAILAB routines:“Laguerp()”， which generates the Laguerre poly- 
nomial (9.9.18);“Gauss1gp()”, which fnds the grid point 帮 "S and the 
coefficient wNi s for Gauss-Laguerre integration formula (5.9.16); and 
“Gauss_Laguerre(f;,N)”，which uses these two routines to carry out 
the Gauss-Laguerre integration method. 


Consider the following integral: 


co co 
ed1t 一 一 ec 
0 0 


， that，Since this integral matches Eq. (5.9.17) with GD) 三 
，Gauss-Laguerre method jls the Tight choice，apply the routine 
0 _Laguerre(f;N)”(manufactured in (a)) with N 一 2 as well as 
the routines“smpsns ()”“adapt_smpsn() “quad() ”and “quad1() 7” 
for evaluating the integral and fll in Table PS.7 with the error Tesults. 
Which turns out to be the best? Is the performance of “quad()” 
improved by lowering the error tolerance?7 
This illustrates that the routine“adapt_smpsn()”Sometimes outperforms the 
MAILLAB built-in routine“quad()”with fewer computations. On the other 
hand, Table P5.7 Shows that it is most desirable to apply the Gauss quadrature 
Schemes only if one of them is applicable to the integration problem. 





+ ce -1 58.0 
0 0 


S.9 Numerical Integrals 


Consider the following integrals. 


CD 记 ”xsinzdxr =1 O) 凡 ximnGinz)dx 一 
1 1 加 
外 册 小 ma 
1 ] 。 四 
人 OA RE 3 


CO) 万 = d 


(9) /xz2e cosxYdx 一 一 


一 (8) /六 Vxe dx = 


Re 
2 


] 


(a) Apply the integration routines“smpsns()”(with N = 104),“adapt 


(b 


NS 


smpsn()” “quad()” “quadl1()” (tol = 10-5) and “Gauss_ leg- 
endre()”(Section 9.9.1) or “Gauss_Laguerre()”(Problem 5.8) (with 
N 一 19) to compnute the above integrals and fll in Table P5.9 with the 
relative errors. Use the upperlower bounds of the integration interval im 
Table P5.9 让 they are Specified in the table. 


Based on the results listed in Table P5.9, answer the following questions 
Or circle the right answer. 
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(人 From the fact that the Gauss-Legendre integration Scheme worked 
best only for (lj), it is implied that the Scheme ls (Tecommendable， 
not Tecommendable) for the case where the integrand function is 
far from being approximated by a polynomial. 


(加 From the fact that the GausS-Laguerre integration Scheme worked 
best only for (9), it is implied that the scheme ils (Tecommendable， 
not Tecommendable) for the case where the integrand function 
excluding the multiplying term els far from being approximated 
by a polynomial. 

(iii) Note the following: 

。 The integrals (3) and (4) can be converted into each other by a 
variable Substitution of x = ZL dx = 一 22 dx. The integrals 
(93) and (6) have the Same relationship. 
。The integrals (7) and (8) can be converted into each other by a 
variable substitution of xz = edx = 一 xz 1d2. 
From the results for (3)-(8), it can be conjectured that the numerical integra- 
tion may work (better,，worse) if the integration interval is changed from [1, co) 
into (0,1] through the substitution of variable like 


X 一 NT dx 二 一 2TotDdu or 一 edx = 一 (007-Ldu (PS.9.1) 


Table P5.9 The Relative Error Results of Applying Various Numerical 
Integration Methods 





Simpson Adaptive Gauss duad quadl 
(N = 109) (tol= 10-9| (N=10) |(tol= 10-9)|(tol = 10-9) 


1.9984e-15 0.0000e 十 00 7.5719e-11 





we | ul 
一， 
[oz 
Fe 


二 大 二 本 
Local Ta 
Ta | 
os Ta | 


5.1077e04| ”| 3Dpsleor 


2.0443e-07 | 


S.10 The BER (Bit Error Rate) Curve of Communication with Multdimensional 
Signaling 
FEor a communication System with multidimensional (orthogonal) Signaling， 
the BER 一 that is, the probability of bit error 一 1 derived as 


2 一 1 Co 
2 ( 三 局 人 (OU-I( 一 V2y -~- VBSNR))e-7 必 】 


二 7 
(P5.10.1) 
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where D is the number of bits，M = 22 is the number of orthogonal wave- 
forms，SNR is the Signal-to-noise-ratio，and CO() is the error function 
defined by 


Co = edy (P5.10.2) 


1 Oo 
V 2 / 
We want to plot the BER curves for SNR = 0:10[dB] and 2 = 1:4. 

(a) Consider the following program “nm5p10.m”，whose objective js to 
compute the values of PCSNR,D) for SNR = 0:10[dB] and 2 = 1:4 
by using the routine“Gauss_Hermite()”(Problem 3.6) and also by 
using the MAILAB built-in routine “quad()”and to plot them versus 
SNR[dB] = 10logio SNR. Complete the incomplete part which com- 


putes the integral in (P5.10.1) over [一 1000, 1000] and run the program 
to obtain the BER curves like Fig. P5.10. 


(b) Of the two routines, which one ls faster and which one presents us with 
more reliable values of the integral in (P3.10.1)? 



































1 0-4 1 1 1 
0 1 2 3 4 5 6 7 SNRI[dB] 9 10 


Figure P5.10 The BER (bit error rate) curvesfor multidimensional (orthogonal) signaling. 


%nm5p10.m: plots the probability of bit error versus SNRbdB 
fs ='Q(-sdqrt(2)*x - sqrt(b*SNR)). (2^b - 1) 
Q = inline(' erfc(x/sqrt(2))/12 ，Xx' )j 
f= inline(fS，Xx'，SNR' ，b' ); 
fex2 = inline([fS '.*xexp(-X.*xX)"]，Xx'，SNR' ，b'); 
0:10; tol = 1e-4; 5 点 SNR[dB] and tolerance Used for "quad ' 
1:4 
2^(b - 1)1(2^b - 1); spi = sqrt(pI); 
1:1ength(SNRdB) ， 
10^(SNRdB(I) 110) ; 
i) = tmp*(1-Gauss_Hermite(f,10,SNR,b)/spi) 
j tmp*(1-quad(fex2,-10,10,tol,[],SNR,b)/spi)i 


了 


了 


Semilogy(SNRdB,Pe，ko' ,SNRdB,Pe1，b+:' ,SNRdB,Pe2, 让 .- )，hold on 
end 
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S.11 Length of Curve/Arc: Superb Harmony of Numerical Derivative/Integral. 


The graph of a function y = jx) of a variable x ls generally a curve and 
its length over the interval [a, p] on the x-axls can be described by a line 
integral as 


六 
人 4= 1 Vadx2 十 dy = 上 | V1 二 (dy/dxz)2 dx 
| VI 二 (CFPoo0)2dx (P5.11.1) 


FEor example, the length of the half-circumference of a circle with the radius 
of unit length can be obtained from this line integral with 


yy 一 FCc) 一 VI 一 2X2， 4 一 一 1， D =1 (P5.11.2) 


Starting from the program “nm5p11.m，make a program that uses the 
numerical integration routines“smpsns()”,“adapt_smpsn()” “quad()”， 
“quadl1()” and “Gauss_Legendre()”to evaluate the integral (P3.11.1,2) 
with the first derivative approximated by Eq. (5.1.8)，where the parame- 
ters like the number of segments (N)，the error tolerance (to1)，and the 
number of grid points (M) are Supposed to be as they are in the pro- 
gram. Run the program with the Step Size 灵 = 0.001, 0.0001, and 0.00001 
in the numerical derivative and fill in Table P3.11 with the errors of the 
results, noting that the true value of the half-circumference of a unit circle 
ls 元 . 


%nm5p11 
= -1;jb=1;55the Lower/upper bounds of the integration interval 


% the error tolerance 
M=205the number of grid points for Gauss-Legendre integration 
IT= piijih=1e-35%true integral and step Size for numerical derivative 
flength = inline(' sqrt(1 + dfp511(x,h).^2) ，Xx'，h');%integrand P5.11.1) 
IsS = Smpsns(flength,a,b,N,h); 
[Ias,points,err] = adapt_smpsn(flength,avb,tol,h); 
Idq = quad(flength,a,b,tol,[],h); 
Idq1 quadl(flength,a,b,tol,[],h); 
IGL Gauss_Legendre(flength,a,b,M,h); 


function df = dfp511(x,h) % numerical derivative of (P5.11.2) 
if nargin < 2，h= 0.001; end 
df = (fp511(x + h)-fp511(x - h))/2/h;i s%Eq.(5.1.8) 


function y = fp511(x) 
y = Sqrt(max(1-X.*X;0)); 5 the function (P5.11.2) 





258 NUMERICAL DIFFERENTIATION/INTEGRATION 


Table P5.11 Results of Applying Various Numerical Integration Methods for 
(P5.11.1,2)/(P5.12.1,2) 


ee Te Te 
IE 
II 


Ta Ts 
| aa ae | | 
[ww Ts aa 
[wa Ta 





S.12 Surface Area of Revolutionary 3-D (Cubic) Object 


The uppeVlower surface area of a 3-D structure formed by one revolution of 
a graph (curve) of a function y = jx) around the x-axis over the interval 
[ac, p] can be described by the following integral: 


记 记 
大 三 2 ydl 一 2 1/ FOODVv1+(CFPGCco))2dx (PS5.12.1) 


For example, the Surface area of a Sphere with the radius of unit length can 
be obtained from this equation with 


= Jo 三 V1 一 如 4 二 一 1 D 一 1 (P5.12.2) 


Starting from the program“nm5p11.m”, make a program“nm5p12.m”that 
uses the numerical integration routines“smpsns()”(with the number of 
Segments N = 1000),“adapt_smpsn()” “quad()” “quadl1()”(with the 
error tolerance tol = 10-6) and“Gauss Legendre()”(with the number 
of grid points M = 20) to evaluate the integral (P5.12.1,2) with the first 
derivative approximated by Eq. (5.1.8), where the parameters like the num- 
ber of segments (N), the error tolerance (tol), and the number of grid points 
(W are Supposed to be as they are in the program. Run the program with 
the step Size 灵 = 0.001, 0.0001, and 0.00001 in the numerical derivative 
and fll in Table P5.11 with the errors of the results, noting that the true 
value of the Surface area of a unit Sphere is 47r . 


S.13 Volume of Revolutionary 3-D (Cubic) Object 


The volume of a 3-D structure formed by one revolution of a graph (curve) 
of a function y = jx) around the x-axlis over the interval [a, pb] can be 
described by the following integral: 


已 
了 一 =/ (xz) dx (P5.13.1) 
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For example, the volume of a Sphere with the radius of unit length (Fig. 
P5.13) can be obtained from this equation with Eq. (P3.12.2). Starting from 
the program “nm5p11.m, make a program “nm5p13.m”that uses the numer- 
ical integration routines“smpsns()”(with the number of Segments N = 
100),“adapt_smpsn()” “quad()” “quad1()”(with the error tolerance 
tol = 10-6)，and“Gauss_Legendre()”(with the number of grid points 
M = 2) to evaluate the integral (P35.13.1). Run the program and fll in 
Table PS5.11 with the errors of the results，noting that the volume of a 
unit Sphere ls 47r/3. 





Figure P5.13 The surface and the volume of a unit sphere. 


S.14 Double Integral 
(a) Consider the following double integral 


2 /区 2 有 
= 上 1 vsnxdare= 人 -yeosxyo=/ 27dy7 =2| =4 
0 J0 0 0 


(P5.14.1) 
Use the routine“int2s()”(Section 9$.10) withM=N= 20,M=N = 
50 andM=N= 100andthe MAILAB built-in routine “dblquad ()” 
to compnute this double integral. Fill in Table P3.14.1 with the results 
and the times measured by using the commands ticy/toc to be taken 
for carrying out each computation. Based on the results listed in 
Table P5.14.1，can we Say that the numerical error becomes Smajller 
as we increase the numbers (M,N) of Segments along the x-axis and 
y-axis for the routine“int2s() ”7 
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(b) Consider the following double integral: 


1 1 1 元 2 
T 一 由 由 dxzdy = 二 (P5.14.2) 
0 0 ] 一 XY 0 


Noting that the integrand function ls Singular at (xz, y) 一 (1 1)，use 
the routine“int2s()”and the MAITLAB built-in routine “db1lquad ()” 
with the upper limit (d) of the integration interval along the y-axis d 
= 0.999, d = 0.9999, d = 0.99999 and d = 0.999999 to compute this 
double integral. Fill in Tables P3.14.2 and P5.14.3 with the results and 
the times measured by using the commands ticy/toc to be taken for 
carrying out each computation , 





Table P5.14.1 Results of 9 | and ae ()”for (P5.14.1) 


人 


2.1649 x 10-8 


ae Ge | 
[汪汪 生 半 和 





Table P5.14.3 ”Results of Running the Double Integral Routine “int2s()”for 
(P5.14.2) 


= 1000， M = 2000， M = 5000， 
1000 N = 2000 N = 5000 


II 





Based on the results listed in Tables P3.14.2 and P3.14.3, answer the 

following questions. 

(iD Can we Say that the numerical error becomes Smaller as we Set the 
Upper limit (d) of the integration interval along the y-axis closer to 
the true limit 12 
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(证 Can we Say that the numerical error becomes Smaller as we increase 
the numbers (M,N) of segments along the x-axis and y-axls for the 
routine“int2s()”? Ithis is contrary to the case of (a)，can you 
blame the weird shape of the integrand function in Eq. (P5.14.2) 
for such a mess-up? 

(cf) Note that the computation times to be listed in Tables P5.14.1 to P5.14.3 
may vary with the Speed of CPU as well as the compnutational jobs which 
are concurrently processed by the CPU. Therefore, the time measured by the 
tic/toc” commands cannot be an exact estimate of the computational load 
taken by each routine. 


S.1S$ Area of a Triangle 


Consider how to find the area between the graph (curve) of a function 矿 (x) 
and the x-axis. For example, let Fox) =xYfor0<x< 和 linorderto fnd 
the area of a right-angled triangle with two equal Sides of unit length. We 
might use either the 1-D integration or the 2-D integration 一 that is，the 
double integral for this job. 


(a) Use any integration method that you like best to evaluate the integral 


1 
] 
= XaX 一 二 (P5.13.1) 
0 2 


(b) Use any double integration routine that you like best to evaluate the 


integral 
1 xD) 1 过 
= 1 loux= 上 1 1dy dx (P5.13.2) 
0 .0 0 .0 


You may get puzzled with Some problem when applying the routine 
“int2s()” 寺 you define the integrand function as 


>>fp515b = inline('1' ，Xx'，y' ); 


It 1 because this function，being called inside the Toutine 
“smpsns_fxy()”, yields just a scalar output even for the vector-valued 
input argument. There are two remedies for this problem. One is to 
define the integrand function in Such a way that it can generate the 
output of the same dimension as the input. 


>>fp515b = inline('1+0x(X+y)，，X'，y')i 


But this will cause a waste of computation time due to the dead multi- 
plication for each element of the Input arguments x and y. The other is 
to modify the routine“smpsns_fxy()”in such a way that it can avoid 
the vector operation. More Specifically，you can Teplace Some part of 
the routine with the following. But, this remedy also increases the com- 
putation time due to the abandonment of vector operation taking less 
time than scalar operation (See Section 1.3). 
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function INTf = Smpsns_fxy(f,x，cdN) 


= Sum_even + f(x,y(n - 1)); 


INTf = (f(x,y(1)) + f(x,y(N+ 1)) + 4*Ssum odd + 2*Ssum_even)*h/13; 





(cf) This problem illustrates that we must be provident to use the vector operation， 
especially in defining a MAILAB function. 


S.16 Volume of a Cone 


Likewise in Section 9$.10, modify the program“nm510.m”So that it uses 
the routines“int2s()”and “dblquad()”to compute the volume of a cone 
that has a unit circle as lts base Slide and a unit height, and run it to obtain 
the values of the volume up to four digits below the decimal point.) 


ORDINARY DIFFEREN TIAL 
EQUATIONS 





Differential equations are mathematical descriptions of how the variables and 
their derivatives (rates of change) with respect to one or more independent 
variable affect each other in a _ dynamical way. Their solutions Show us how 
the dependent variable(S) will change with the independent variable(S). Many 
problems in natural Sciences and engineering fields are formulated into a scalar 
differential equation or a vector differential equation 一 that 1Ss,，a System of dif- 
ferential equations. 

In this chapter, we look into Several methods of obtaining the numerical solu- 
tions to ordinary differential equations (ODEs) in which all dependent variables 
(x) depend on a single independent variable (1). First, the initial value problems 
(LVPSs) will be handled with several methods including Runge- 下 utta method and 
predictor-corrector methods in Sections 6.1 to 6.$. The final section (Section 6.0) 
will introduce the Shooting method and the finite difference method for solving 
the two-point boundary value problem (BVP). ODEs are called an IVP if the 
values XY( 四 ) of dependent variables are given at the initial point 如 of the inde- 
pendent variable, while they are called a BVP 让 the values xz(io)/ x(r) are given 
at the initial/final points 如 and x7. 


6.1 EULER'S METHOD 


When talking about the numerical solutions to ODESs，everyone starts with the 
Eulers method, since it is easy to understand and simple to program. Even though 
its low accuracy keeps it from being widely used for solving ODES, it gives us a 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang,，Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 儿 Sons, Inc. 
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clue to the basic concept of numerical solution for a differential equation Simply 
and clearly. Let's conslider a first-order differential equation: 


y (GD) 十 ayYCGD 一 7 with y(0) = 一 y0 (6.1.1) 


It has the following form of analytical solution: 
FA 二 0 人 
y(D) 王 (mw--)。 闭 : 一 (0.1.2) 
Q Q 


which can be obtained by using a conventional method or the Laplace trans- 
form technique [ 玫 -1，Chapter 5]. However, such a nice analytical Solution does 
not exlst for every differential equation; even if it exists，it 18 not easy to 
fnd even by using a _ computer equipped with the capability of Symbolic com- 
putation. That is why we Should study the numerical solutions to differential 
equations. 

Then, how do we translate the differential equation into a form that can eas- 
jly be handled by computer? First of all，we have to replace the derivative 
y (0 =ady/drinthe differential equation by anumerical derivative (introduced in 
Chapter 3)，where the Step-size /jls determined based on the accuracy require- 
ments and the computation time constraints. Eulers method approximates the 
derivative in Eq. (6.1.1) with Eq. (9.1.2) as 


74 十 /一 7>(D) 
忆 
yGL 十 由) 一 (1 一 ap)y( 十 Pr with y(0) = y0 (0.1.3) 


十 Gy 三 7 


and solves this difference equation Step-by-step with increasing 1 by /each time 
from 上 = 0. 


y(1) = (1 一 a1)y(O) 十 j7 
yO 人 = (1 一 ay(OD 十 厅 
y(3 由 = (1 一 ajy(C21) 十 1 一 (1 一 aj)3yo 十 并 一 G1)777 


(1 一 Ga1)y0 十 77 
(] 一 Cj)2y0 十 (1 一 Ca1)jr 十 1 (0.1.4) 


This ls a numeric sequence {y(K1)}，which we call a numerical solution of 
了 上 q. (6.1.1). 

To be Specific, let the parameters and the injtial value of Eq. (6.1.1) be a = 1， 
7 一 1 and yo =0. Then, the analytical Solution (6.1.2) becomes 


yG 一 1 一 er4 (6.1.5) 
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snm610: Euler method to Solve a 1st-order differential equation 
Clear，CLf 
a=1ir=1iyo=0i tf=2; 
t= [0:0.01:tfl]j yt= 1 - exp(-axrt); %Eq.(6.1.5): true analytical SolLution 
plot(t,yt，k')，hold on 
klasts = [8 4 2]; hs = tf./klastsi 
y(1) = y0i; 
for itr = 1:3 %wWwith Various step Size h = 1/18,1/14,1/2 
kJlast klasts(itr); h = hs(itr);i y(1)=y0; 
for K 1:k1last 
y(k+1)= (1 - arh)*y(k) +hxri %5Eq.(6.1.3) : 
plot([K - 1 kj*h[y(k) y(k+1)]，b ，k*shy(k+t1)，ro ) 
if k < 4，pausej end 
end 
end 





and the numerical solution (6.1.4) with the Step-size 灵 =0.3 and 灵 一 0.23 are 
as listed in Table 6.1 and depicted in Fig. 6.1. We make a MAILAB program 
“nm610.m"，which uses Eulers method for the differential equation (6.1.1), actu- 
ally solving the difference equation (6.1.3) and plots the graphs of the numerical 
Solutions in Fig. 6.1. The graphs seem to tell us that a Small Step-size helps 
ITeduce the error So as to make the numerical solution closer to the (true) ana- 
lytical solution. But, as will be investigated thoroughly in Section 60.2, it is only 
partially true. In fact a too Small step-size not only makes the computation time 
longer (proportional as 1/j),but also results in rather larger errors due to the 
accumulated round-off effect. This is why we Should look for other methods to 
decrease the errors rather than Simply reduce the step-Slze. 

Eulers method can also be applied for solving a first-order vector differential 
equation 


y() 一 fy) with y(1o) = y0 (6.1.6) 


which is equivalent to a high-order scalar differential equation. The algorithm 
can be described by 


ykH 三 史 十 7E(,yb) WwWith y(1o) = y0 (0.1.7) 


Table 6.1 ANumerical Solution of the Differential Equation (6.1.1) Obtained by the 
Eulers Method 


y(0.25) = (1 一 am 十 杂 =1/4 = 0.25 
y(0.50) = (1 一 ay 十 1r = 1/2= 0.5 y(0.50) = (3/4)y(0.25) 十 1/4 = 0.4375 
y(0.75) = (3/4)y(0.50) + 1/4 = 0.5781 


y(1.00) = (1/2)y(0.3)+ 1/2 = 3/4 = 0.75 | yG.00) = (3/4)y(0.75) 上 + 1/4 = 0.6836 
y(1.25) = (3/4)y(1.00) + 174 = 0.7627 
y(1.50) = (1/2)y(1.0) + 1/2 = 7/8 = 0.875 | yd.50) = (3/4)y(1.25) 上 + 114 = 0.8220 
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0.8 上 
0.6 上 
0.4 
the (true) analytical solution 
y(=1 一 ee 
0.2 浊 
0 1 | 
0 0.5 1 1.5 [ 2 


Figure 6.1 Examples of numerical solution obtained by using the Eulers method. 


and is cast into the MAILAB routine“ode_Euler() 


function [t,y] = ode_Euler(f,tspan,yO,N) 
#%EUJer 'Ss method to Solve Vector differential equation y'(t) = flt,y(t)) 
#%for tspan = [to,tf] and with the Initial value y0O and N time Steps 
if nargin<4 | N<= 0,，N=100; end 
if nargin<3，y0 = 0; end 
h (tspan(2) - tspan(1))/VNi %stepsize 
坏 tspan(1)+[O:N] *h;ji %time Vector 
y(1,:) = y0(:) ; %5alwaySs make the initial value a row Vector 
for k= 13:N 
y(k+ 1,:) = y(k,:) +hx*feval(f,t(k),y(k,:)); %5Eq.(6.1.7) 
end 





6.2 HEUN'S METHOD: TRAPEZOIDAL METHOD 


Another method of solving a first-order vector differential equation like Eq. (6.1.0) 
comes from integrating both sides of the equation. 


y (GD) = fo, y)， yy) 人 一 yGkHD 一 yx) = |/ 人 1, y) di 


灰 


yGuD 一 yd 二 1 fed with ydo 一 (6.2.1) 


大 


H 开 we assume that the value of the (derivative) function ft,y) is constant 
as f( 太 ,y( 姑 )) within one time step [ 灰 克 +1)，this becomes Eq. (6.1.7) (with 不 一 
如 +1 一 灵 ), amounting to Eulers method. 开 we use the trapezoidal rule (5.3.3), 二 
becomes 


万 
yk+H1 一 了 十 了 人 tx， yk) 十 fl yk+HTD)} (0.2.2) 
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function [t,y] = ode_Heun(f,tspan,y0O,N) 
#%Heun method to Solve Vector differential equation y'(t) = flt,y(t)) 
%5 for tspan = [to,tf] and with the initial value y0 and N time Steps 
if nargin<4 | N<= 0,，N= 100; end 
if nargin<3，y0 = 0j; end 
(tspan(2) - tspan(1))/N; %stepSlize 
tspan(1)+[O:N] '*h; %time Vector 
(1,:) = y0(:) '; %5always make the initial Value a row Vector 
for kK= 13:N 
fk = feval(f,t(k),y(k,:)); y(k+1,:) = y(k,:)+hx*fk;i %Eq.(6.2.3) 
y(k+1,:) = y(k,:) +h/2*(fk +feval(f,t(k+1),y(k+1,:))); *%Eq. (6.2.4) 
end 


h 
七 
y 





But, the right-hand side (RHS) of this equation has yk+1，which ls unknown at 
灰 . To resolve this problem，we replace the yk+l on the RHS by the following 
approximation: 

ykH 兰 y 十 /1( 大 yb) (0.2.3) 


So that lt becomes 


万 
ykH 三 yk 十 了 姓 人 tx， yk) 十 fr, ye 十 ECG, yb))} (0.2.4) 


This is Heuns method，which is implemented in the MAILAB routine 
“ode_Heun() .Iti a kind of predictor-and-corrector method in that lt predicts 
the value of yk+l1 by Eq. (6.2.3) at 灰 and then corrects the predicted value by 
Eq. (6.2.4) at 太 +1. The truncation error of Heun's method is O(P22) (proportional 
to 12) as shown in Eq. (5.6.1D), while the error of Eulers method is O(P). 





6.3 RUNGE-KUTTA METHOD 


Although Heun's method ls a little better than the Eulers method, it is Still not 
accurate enough for most real-world problems. The fourth-order Runge-Kutta 
(RK4) method having a truncation error of O(P4) is one of the most widely used 
methods for solving differential equations. Its algorithm is described below. 


灵 
yk+H1l 三 yk 十 半 de 十 2fx2 十 2fx3 十 也 4) (0.3.1) 
Where 
仅 1 三 攻克 , yb) (6.3.2a) 
人 2 三 三 大 十 有 /2 yk 十 长 17/2) (6.3.2b) 
fs3 三 在 克 十 用 /2 yk 十 公 27/2) (0.3.2c) 


fa4 三 下 大 十 天 ,yc 十 约 37) (0.3.2d) 
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function [t,y] = ode_RK4(f,tspan,y0,N,varargin) 
%5Runge-Kutta method to Solve Vector differential eqn y'(t) = flt,y(t)) 
s# for tspan = [to,tf]l and with the initial value y0O and N time steps 
if nargin <41N<= 0 N= 100; end 
if nargin < 3，y0 = 0j end 
y(1，:) = y0(:) ; %make it a row vector 
h = (tspan(2) - tspan(1))/Ni 七 = tspan(1)+[O0:N] *h; 
for kK = 13:N 
f1 hx*feval (k,:),varargin{t:}); fl = fl1(:) 
f2 hx*feval h/2,y(k,:) + fl1/2,varargin{f:}); 
f3 hx*feval h/2,y(k,:) + f2/2,varargin{f:}); 
f4 hx*feval h,y(k,:) + f3,varargin{ft:}); Tf4 


end 


snm630: Heun/Euer/RK4 method to Solve a differential equation (d.e.) 
Clear，ClLf 

tspan = [0 2]; 

t = tspan(1)+[0:100]*(tspan(2) - tspan(1))/1100; 

a=1iyt= 1 -exp(-ax*t); %5Eq.(6.1.5): true analytical Solution 
plot(t,yt，k')，hold on 

df61 = inline('-y+1，t，y'); s%Eq.(6.1.1): d.e. to be Solved 
yY0O0=0j N= 4 

[tl1,yel = 0ed_Euler(df61,tspan,y0O,N) ; 

[tl1,yh] = ode_Heun(df61,tspan,y0,N) ; 

[t1,yr]l = 0ode_RK4(df61,tspan,y0O;N) ; 
plot(t,yt，Kk'，tl,ye，'b:'，tlyh，b:' ，tlyrr: ) 
plot(t1,ye，bo' ，tl,yh，b+' ，t1yr，rx* ' ) 

N = 1e3; %to estimate the time for N iterations 

tic，[t1,yel] ode_Euler(df61,tspan,y0,N); time_EUlLer = toc 
tic，[t1,yhj] ode_Heun(df61,tspan,y0,N); time_Heun = toc 
tic，[t1,yr] ode_RK4(df61,tspan,y0,N); time_RK4 = toc 





Equation (0.3.1) is the core of RK4 method，which may be obtained by sub- 
stituting Simpson's rule (9.5.4) 


| 2XkHL 一 克 大 


jx) dx 三 本 (大 十 4 友 HP 十 态 四 with 肠 一 0 

(0.3.3) 
into the integral form (6.2.1) of differential equation and replacing 太 H172 with 
the average of the Successive function values (jz 十 加 3)/2. Accordingly,， the 
RK4 method has a truncation error of O(1p4) as Eq. (5.6.2) and thus is expected 
to work better than the previous two methods. 

The fourth-order Runge- 下 utta (RK4) method is cast into the MAITLAB rou- 
tine“ode_RK4()”. The program “nm630.m” uses this routine to Solve Eq. (0.1.1) 
WwWith the step Size 7 = 人 rr 一 1)/N=2/4=0.3 and plots the numerical result 
together with the (true) analytical solution. Comparison of this result with those of 
Eulers method ("“ode_Euler()”) and Heun's method (oode_ Heun()” is given in 
Fig. 0.2, which shows that the RK4 method is better than Heun's method, while 
Eulers method ls the worst in terms of accuracy with the Same step-size. But， 


灰 
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y(D=1 一 er 
the (true) analytical solution 






Runge-Kutta solution 米 二 
Heun solution 十 











Figure 6.2 ” Numerical solutions for afirst-order differential equation. 


in terms of computational load, the order ls reversed，because Eulers method， 
Heun's method, and the RK4 method need 1, 2, and 4 function evaluations (calls) 
per lteration, respectively. 


(cf) Note that a function call takes much more time than a multiplication and thus the 
number of function calls should be a criterion in estimating and comparing compu- 
tational time. 


The MAILAB built-in routines“ode23()”and“ode45()”implement the 
Runge-Kutta method with an adaptive Step-Size adjustment，which uses a 
large/small step-size depending on wphether j(O) 1 Smooth or Tough.， 了 
Section 0.4.3，we will try applying these Toutines together with our routines to 
solve a differential equation for practice rather than for comparison. 


6.4 PREDICTOR-CORRECTOR METHOD 


6.4.1 Adams-Bashforth-Moulton Method 


The Adams-Bashforth-Moulton (ABM) method consists of two StepSs. The first 
Step ls to approximate fit,y) by the (Lagrange) polynomilal of degree 4 matching 
the four points 


{( 3 入 -3)， (2 公公 (大 信 )} 





and substitute the polynomial into the integral form (6.2.1) of differential equation 
to get a predicted estimate of yk+1. 


太 
pk+l 一 yk 十 人 5D)dL 三 了 十 二 作 9%-3 十 37f-2> 一 59f 1 十 55 人) 
0 
(0.4.1a) 
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The second step is to repeat the Same work with the updated four points 


{( 丰 -2 入 2)，( 丰 -5 帮 -D)，( 丰 , 引 )， (HT 和 TD Ge 三 攻克 plD) 








to get a _ corrected estimate of yk+1， 


户 万 
CH 一 了 + 13(0)G1 一 yK 十 二 全 - 一 Sf_1 十 19 人 十 9f+) (6.4.1b) 
0 


The coefficients of Eqs. (6.4.1a) and (6.4.1b) can be obtained by using the 
MAILAB routines“lagranp()”and“polyint()”，each of which generates 
Lagrange (coeffcienb polynomials and integrates a polynomial，respectively. 
Let s try running the program “ABMc .m”. 


>>abmc 
CAP = -3/8 37/24 -59/24 55/24 
CAC = 1/24 -5/124 19/124 3/18 


%ABMc .m 
#% Predictor/Ccorrector coefficients in Adams-Bashforth-Moulton method 
ClLear 
format rat 
[1,L] lagranp([-3 -2 -10],[0000]); s%only coefficient polynomial L 
for mm 13:4 

工 L polyint(L(m,:)); s%5indefinite integral of polynomial 

CAP(m) = polyval(IiL,1)-polyval(IL,0); %5definite integral over [0,1] 
end 
CAP %5Predictor coefficients 
[1,L] = lagranp([-2 -10 1],[0000]); s%5only coefficient polynomial 上 L 
for 13:4 

工 L polyint(L(m,:)); %indefinite integral of polynomial 

cAC(m) = polyval(iL,1) - polyval(iIL,0); %definite Integral over [0,1] 
end 
CAC s%5Corrector coefficients 
format Short 





Alternatively，we write the Taylor series expansion of yk+1l about 灰 and that 
of yk about 灰 +1 as 

1 ]3 

yk+H1 三 Y 十 At 十 万 八 十 相 


f) 十 包 1G) 十 和 fg 十 …. (6.4.2a) 
人 7 
和 太 3 太 4 万 5 
(2) (3) (4) 
鸡 二 ye 一 At 十 万 人 一 可 而 人 er 一 本 十 
万 2 太 3 太 4 帮 5 
y (2) (3) (4) 


and replace the first ,second, and third derivatives by their difference approxi- 
mations. 
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ykH 三 允 十 ] 人 十 一 
2 


1 ( 涯 3 十 46 > 一 S 仅 二) 


和 ( - 吾 3 十 38 一 38 1 十 总 人 + 





下 人 12 


14 7 一 3f 2 一 3f 十 作 3 放 
( 王 汪 汪 3) 各 





SR 13 120 大 
一 由 十 2 (一 9f，3 十 37f 59f， 1 十 S5f) 十 全 175f() 
二 yYK 二 大 一 3 大 一 2 有 -1 天 07 
(6.4.1a) 291 
久 ”prl 十 本 (6.4.3a) 


712 7 1 3 一 卫 1 
人 地 ( 3 从 -2 十 二 3 人 十 志 公 +1 If 二 


13 /一 人 2 十 4f 1 一 5 和 十 2 11 
和 (和 


万 2 12 
14 1/ -fa 二 38 1 一 3 十 了 3 及 
( 卫 f 十. 


万 3 2 120 4 


一 y 二 冯 训 一 SB Il 十 19f 十 9%8 三 2 的 
大 5 大 一 2 大 一 1 天 K 十 1 720” k+1 


CA 19 ,5 
生疏 (6.4.3D) 


These derivations are Supported by running the MAILAB program“ABMc1 .m”. 


%ABMc1 .m 

%another Way to get the ABM coefficients together with the error ternm 
Clear，Tformat rat 

for TIL = 1:3，[cierril = difapx(Ii,[-3 0]);j c(i,:) = cij err(Ii) = errii 


end 
cAP = [0001]I+[1/2 1/6 1/24]*c，errp = -[1/2 1/6 1/24]*err' + 1/1120 
cAC = [0001I+[-1/2 1/6 -1/24]*c，errc = -[-1/2 1/6 -1/24]*err' + 1/120 


format Short 





From these equations and under the assumption that 的 天 和 兰 玉 ,we can 


Write the predictorcorrector errors 3aS 
2531 


251 
瓦 兰 一 pbi 和 一 71 全 一 及 75 6.4.4 
Pi 二 yt 一 ptrlS 0 人 7 人 al) 











19 19 
书 一 -col 名 一 一 715 人 宕 一 一 一 开 15 6.4.4b 
CH 三 yiH1 一 cl 0 全 也 + 1 ( ) 
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We still cannot use these formulas to estimate the predictorcorrector errors, Since 
K ls unknown. But, from the difference between these two formulas 


270 270 270 
玉 15 三 





天 PH 一 下 ctH 三 CH 一 ptHl 位 7T20 = 75T2Ptl 二 尼 c.k+Hl 
(0.4.3) 
we can get the practical formulas for estimating the errors as 
尼 PiH 三 ykHl 一 pk+l 宕 2 一 pk+1) (0.4.0a) 
270 
下 ctHl 三 yk+l 一 ceHl 兰 0 一 pk+1) (6.4.6b) 
， 270 











These formulas give us Tough estimates of how close the predicted/corrected 
values are to the true value and So can be used to improve them as well as to 
adjust the step-Ssize. 


2531 
pk+l 一 ptH 十 二 (ce 一 pb 一 me (0.4.7a) 
270 
( ) 一 (0.4.7b) 
于 全 入 全 As | | .个 
CE 二 1 Cl 270 CE+HL 一 Pk+1l yYK 二 1 








These modiftication formulas are expected to Teward our efforts that we have 
made to derive them. 

The Adams-Bashforth-Moulton (ABM) method with the modification formu- 
las can be described by Eqs. (6.4.1a),， (6.4.1pb), and (6.4.7a), (6.4.7b) summarized 
below and is cast into the MAILAB routine“ode_ABM()”. This scheme needs 
only two function evaluations (calls) per iteration，while having a truncation 
error of O(1ps) and thus is expected to work better than the methods discussed so 
far. It is Implemented by the MAILAB built-in routine“ode113()”with many 
additional sophisticated techniques. 


(Adams-Bashforth-Moulton method with modification formulas) 
万 
Predictor: perl = yk 十 世人 和 十 37 公 -> 一 9 有 -1 十 3 人) (6.4.8a) 
. 2531 
Modifier: mx+l = 一 pk+l 十 0(c 一 po) (6.4.8b) 
屡 
Corrector: Ck+l 三 YXK 十 二 作 - 人 f_ 1 十 19f 人 十 9f(A 1， mk+1)) (0.4.8c) 








19 
yk+H1 三 ck+Hl 一 二 0 Ce 一 pk+D) (0.4.8d) 
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function [t,y] = ode_ABM(f,tspan,y0;,N,KC,varargin) 
#%Adams-Bashforth-Moulton method to Solve Vector d.e. y'(t) = flt,y(t)) 
sfor tspan = [totf]l and with the initial value y0 and N time Steps 
s5 USing the modifier based on the error estimate depending on KC = 1/0 
if nargin < 5，KC = 1; end %with modifier by default 
if nargin <4 |N<=0,，N=100; ends%default maximum number of iterations 
y0 = y0(:) ; %make it a row vector 
(tspan(2) - tspan(1))/N; %step Size 
tspan0 = tspan(1)+[0 3]*h; 
[各 5 人 = We tspan0,y0,3 1Varargant }); s%initialize by Runge-Kutta 
t(1:3)”， Tt(4) :hitspan(2)] 
for = 1: 4，F(k,:) = feval(fit(k) ,y(k,:) ,vararginf:)); end 
p = Y(4，:) 3 C = y(4,:)j KC22 = KC*251/270; KC12 = KC*19/1270j 
h24 = 2 h241 = h24*[1 -5 19 9]; h249 = h24*[-9 37 -59 55]; 
for k = 
p1 人 +h249*F; %Eq.(6.4.8a) 
m1 pk1 + KC22*(Cc-p); ssEq.(6.4.8b) 
C1 y(K，: )+ 
h241*[F(2:4,:); feval(f,t(k + 1),ml,varargin{f:})]; ssEq.(6.4.8c) 
y(k+ 1,:) = cl - KC12*(c1 - p1); s%Eq.(6.4.8d) 
= pl1j CC= cli %update the predicted/corrected values 
= [F(2:4,:); feval(f,t(k+ 1),y(k + 1,:),varargin{:})]; 





6.4.2 Hamming Method 


function [t,y] = ode_Ham(f,tspan,y0,N,KC,varargin) 
点 Hamming method to Solve Vector d.e. y'(t) = flt,y(t)) 
sfor tspan = [totf]l and with the initial value y0 and N time Steps 
s USing the modifier based on the error estimate depending on KC = 1/0 
if nargin < 5，KC = 1; end %with modifier by default 
if nargin <4 |N<=0,N= 100; end %default maximum number of iterations 
if nargin < 3，y0 = 0j end %5default initial Value 
y0 = y0(:) ;end %make it a row Vector 
(tspan(2)-tspan(1))/N; %step Size 
tspan0 = tspan(1)+[0 3]*h; 
[t,y] ode_RK4(f,tspan0,y0,3,Vvarargin{:}); %Initialize by Runge-Kutta 
:3)”， t+t(4):h:tspan(2)] ; 


):) = feval(f,t(k),y(k,:),varargin{f:}); end 
:); h34 = h/3*4; KC11 = KC*112/1121; KC91 = KC*9/1121; 
h312 = 3*hx[-1 2 1]; 
4:N 


y(k - 3,:) + h34*(2*(F(1,:) + F(3,:)) - F(2,:)); %5Eq.(6.4.9a) 
p1 + KC11*(c - p); s%5Eq.(6.4.9b) 
(-y(k - 2，:) + 9*y(k:) + 
h312*[F(2:3,:); feval(f,t(k + 1),ml,varargin{t:})])/18; %Eq.(6.4.9c) 
= Cl1 - KC91*(c1 - p1); %Eq. (6.4.9d) 
C= cli s%update the predicted/corrected Values 
:); feval(f,t(k+ 1),y(k + 1，:)，,varargin{f:})]; 
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(Hamming method with modification formulasy》 


47 
Predictor: pril 三 yk3 十 本 一 公 _1 十 2f) (6.4.9a) 


. 112 
Modifier: mk+l = 王 piel 十 Ttcx 一 pb (6.4.9b) 


1 
Corrector: ck+1l 一 gt2yx 一 V-2 十 31( 一 公 -1 十 2 和 十 不 pr mei)) (0.4.9c) 





9 
yk+1 三 ck+1 一 1 (ce 一 pk+l) (0.4.9d) 





In this section，we introduce just the algorithm of the Hamming method [H-H] 
Summarized in the box above and the corresponding routine“ode_Ham()”, which 
is another multistep predictor-corrector method like the Adams-Bashforth 一 
Moulton (ABM) method. 

This scheme also needs only two function evaluations (calls) per iteration， 
while having the error of O(1s5) and so is comparable with the ABM method 
discussed in the previous Section. 


6.4.3 Comparison of Methods 


The major factors to be considered in evaluating/comparing different numeri- 
cal methods are the accuracy of the numerical Solution and its computation 
time. In this section，we will compare the routines“ode_RK4()”“ode_ABM()”， 
“ode_Ham()”,“ode23()”“ode45() ”and“ode113()”by trying them out on 
the same differential equations, hopefully to make some conjectures about their 
performances. It is important to note that the evaluation/comparison of numer- 
ical methods is not So Simple because their performances may depend on the 
characteristic of the problem at hand. It should also be noted that there are other 
factors to be conslidered，Ssuch as Stability，versatility，proof against run-time 
error，and So on. These points are being considered in most of the MAILAB 
built-in routines. 

The first thing we are going to do is to validate the effectiveness of the mod- 
ifers (Eqs. (6.4.8b,d) and (6.4.9b,d)) in the ABM (Adams-Bashforth-Moulton) 
method and the Hamming method. For this job， we write and run the program 
“nm643_1.m” to get the results depicted in Fig. 6.3 for the differential equation 


y=-y0D+1 with y0O)=0 (6.4.10) 


which was given at the beginning of this chapter. Fig. 06.3 Shows us an interesting 
fact that， although the ABM method and the Hamming method，even without 
modifiers, are theoretically expected to have better accuracy than the RK4 (fourth- 
order Runge-Kutta) method, they turn out to work better than RK4 only with 
modifiers. Of course, it is not always the case, as illustrated in Fig. 0.4，which 
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Figure 6.3 _ Numerical solutions and their errors for the differential equation y 人 = 一 Y(GD 十 1. 
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Figure 6.4 Numerical solutions and their errors for the differential equation y( = yY(bD 十 1. 
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we obtained by applying the same routines to solve another differential equation 


y (一 yGD) 十 1 with y(0) = 0 (0.4.11) 
where the true analytical solution 1S 
yOD=e 一 1 (6.4.12) 


s%nm643_1: RK4/Adams/Hamming method to Solve a differential eq 
Clear，Clf 
t0=0itf= 10; y0 = 0; %starting/final time，initial value 
N = 50; %number of Segments 
df643 = inline('-y+1'，t'，y') 
f643 = inline('1-exp(-t)'， 七 ') 
for KC = 0:1 
tic，[t1,yR] = ode_RK4(df643, [to tf],y0,N); tR = toc 
tic，[t1,yAI] ode_ABM(df643, [to tf],y0,N,KC); tA = toc 
tic，[t1,yH] ode_Ham(df643, [to tf],y0;N,KC); tH = toc 
yt1 = f643(t1); %true analytical Solution to plot 
Subp1lot(221 + KC*2) s%plot analytical/numerical Solutions 
plot(t1,yt1，'k'，t1,yR,'k'，t1,yA'k--'，tliyH，k:) 
tmp = abs(yt1)+eps;j 1 _ tl1 = length(t1); 
eR = abs(yR - yt1)./tmp;i e_R=norm(eR)/1lLt1i 
eA = abs(yYA - yt1)./tmp;j e_A=norm(eA) /1lLt1i 
eH abs(yH - yt1)./tmp; e_H=norm(eH)/1lLt1i 
Subp1lot(222 + KC*2) %plot relative errors 
plot(t1,eR，Kk'，t1,eA，k--'，tl1，eH，k: ') 


%differential equation to SolLve 
%true analytical Solution 


学 
了 


s%nm643 2: 0de23()/ode45()/ode113() to Solve a differential ed 
Clear，ClLf 

t0=0ojitf= 10;jy0=0jN=50; s%starting/final time，initial value 
df643 = inline(y+ 1'，t'，y'); *%wdqifferential equation to SolLve 

f643 = inline('exp(t) - 1 ，t'); s%true analytical Solution 
tic，[t1,yYR] = 0ode_RK4(df643, [to tf]l,yo,N); time(1) = toci 
tic，[t1,yYA] = 0ode_ABM(df643,[t0O tf]l,yoO;N); time(2) = toc; 

yt1 = f643(t1) 

tmp = abs(yt1)+ eps;j 1 tl = Jength(t1) ; 

eR abs(yR-yt1)./tmp;j err(1) = norm(eR)/1L ti1; 

eA = abs(yA-yt1)./tmp;i err(2) = norm(eA)/]1 tfi 

options = 0deset('` RelTol ,1e-4); %5Sset the tolerance of relative error 
tic，[t23,yode23] = ode23(df643, [to tf]l,y0,options); time(3) = toci 
tic，[t45,yode45] = 0ode45(df643, [to tf]l,y0,options); time(4) = toci 
tic，[t113,yode113] = ode113(df643, [to tf],y0,options); time(5) = tocij 
yt23 = f643(t23); tmp = abs(yt23) + eps; 

eode23 = abs(yode23-yt23)./tmp;j err(3) = norm(eode23)/1Length(t23) ; 
yt45 = f643(t45); tmp = abs(yt45) + eps; 

eode45 = abs(yode45 - yt45)./tmp; err(4) = norm(eode45)/1Length(t45) ; 
yt113 = f643(t113); tmp = abs(yt113) + epsij 

eode113 = abs(yode113 - yt113)./tmp;i err(5) = norm(eode113)/JLength(t113) ; 
Subplot(221)，plot(t23,yode23，Kk'，t45,yode45，b ，t113,yode113，m ) 
Subplot(222)，plot(t23,eode23，kKk'，t45,eode45，b-- ，t113,eode113，F: ) 
err，time 
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Table 6.2 ”Results of Applying Several Routines to solve a Simple Differential Equation 
ode_RK4() ode_ABM() ode_Ham() ode231() ode451() ode1131() 


Relative error ”0.0925 x 10-* 0.0203 x 10-4 0.0179 x 10-4 0.4770 x 10-* 0.0422 x 10-4 0.1249 x 10- 
Computing time 0.05 Sec 0.03 sec 0.03 Sec 0.07 Sec 0.05 sec 0.05 Sec 


Readers are invited to Supplement the Program“nm643_2.m” in Such a way 
that“ode_Ham()”is also used to solve Eq. (6.4.11). Running the program yields 
the results depicted in Fig. 6.4 and listed in Table 6.2. From Fig. 0.4, it is note- 
worthy that, without the modifiers, the ABM method seems to be better than the 
Hamming method; however, with the modifiers, it is the other way around or at 
least they run a neck-and-neck race. Anyone will see that the predictor 一 corrector 
methods such as the ABM method (ode_ABM()) and the Hamming method 
(ode_Ham()) give us a better numerical solution with less error and shorter com- 
putation time than the MAILAB built-in routines“ode23()”,“ode45()”， and 
“ode113()”as well as the RK4 method (ode_RK4()), as listed in Tabple 6.2. But， 
a general conclusion Should not be deduced just from one example. 


6.5 VECTOR DIFFERENTIAL EQUATIONS 


6.5.1 _ State Equation 


Although we have tried using the MAILAB routines only for scalar differential 
equations，all the routines made by us or built inside MAILILAB are ready to 
entertain first-order vector differential equations, called state equations, as below. 


2 一 万 (xx ) with xz1(10) 三 X10 


X2 (1) 一 户 (,X10),x2(0D，) With x2(10) 一 X20 


X' (1) 一 fr, xD)) with X(1 加 0) 一 X0 (6.3.1) 
For example， we can define the System of first-order differential equations 


X1 (CD 一 2 人 with xl1(0) 一 1 
(0.5.2) 
X2 (1 一 一 z2( 十 1 with x2(0) 一 一 1 


in a fle named“df651.m”and solve it by running the MAILAB program 
“nm651_1.m>，which uses the Toutines “ode_Ham()”/ode45()”to get the 
numerical Solutions and plots the results as depicted in Fig. 0.3. Note that the 
function given as the first input argument of “ode45()”must be fabricated to 
generate jlts value in a column vector or at least in the same form of vector as 
the input argument 'x” So long as it ls a vector-valued function. 
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s%nm651 1 to Solve a System of differential eqs.，1i.e.，state equation 

df = 'df651 

t0=0Oiltf=2;x= [1 -1]; %start/final time and initial Value 

N = 45; [tH,xH] = ode_Ham(df, [to tf],xo,N); %with N = number of Segments 


[t45,x45] = 0ode45(df,[to tf],x0) ; 
plot(tH,xH)，hold on，pause，plot(t45,X45) 


function dx = df651(t,X) 
dx = zeros(Size(Xx)); %row/column vector depending on the Shape of X 
dx(1) = X(2)) dx(2) = -X(2) + 1 





Especially for the state equations having only constant coefficients like Eq. 
(0.3.2)，we can change it into a matrix-vector form as 


xz 人 | 0 1 工 TxzD 0 
区 浊 轩 攻 本 | 区 赤 | |wo (6.5.3) 
with 加 光 = 本 andu(D) =1IVi>0 


X' (1) 一 4X() 十 刀 U(G) with the initial state X(0) and the input x() (6.5.4) 


which is called a linear time-inyvariant (LID state equation, and then try to find 
the analytical solution. For this purpose，we take the Laplace transform of both 
Sides to write 


SX(S) 一 X(O0) 一 4X(G) 十 BUG) with XG) 三 ZX GD) 三 ZXCD} 


[S7 一 4]XG) = xO) 十 BUG)，XG) = [97 -4]-IxOD+IT -4]-1BUG) 

(0.3.5) 
where ZL{fxz()} and 二 -1X(C)}) denote the Laplace transform of x(D) and the 
inverse Laplace transform of X(s), respectively. Note that 


[7 一 4 有 = 一 4 = 上 4 二 42 十 | 


(= 一 ZL 一 4 (6.5.6) 
4? 43 
和 with (0) 一 7 


By applying the convolution property of Laplace transform (Table D.2(4) in 
Appendix D) 


ZHIs7 一 4]-1BUG) = 一 4]*Z-IHBUG) = 四 *Bu(C) 


= 三 0 一 让 Ba(r)dreT 4 一 zBu(r)dr (6.5.7) 
四 5 
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we can take the inverse Laplace transform of Eq. (6.3.5) to write 
X(C) 一 少 (1)XO) 十 0 *BL 三 轴 ()X(O) 十 人 gb 一 T)BxT)dr (0.3.8) 
0 


For Eq. (6.35.3)，we use Eq. (6.3.6) to find 


bg 一 ZL-4]7) 


站 


| 1 es ,|| 
四 GCT+THDL 0O yyY 


-二 | 宅 和 鲍 。 | (65.9) 





0 1/G+D 0 ce 


and use Eqs. (0.3.8), (6.3.9), and x() 一 (人 二 1Y1L>0Otoobtan 


1 1-e|f1l 1 1 一 ev 5951T0 
oj 
丰 
1 一 1 十 2e- 
引 下 | (6.5.10) 


2 一 示 三 : 2 一 (4 一 ID) 
二 一 e 二 二 2 一 (一 D 吕 
0 


Alternatively，we can directly take the inverse transform of Eq. (6.5.3) to get 





X(G)=[S7 一 4]-IxO)+[7 一 4]-BUG)} 


| 


到 和 8 有 1 2 十 1 


2 十 1 1 1 2 
证 xi 一 1 上 一 1 二 2e- (6.5.12a) 
S2(C8 十 1D) SS 3 3 二 1 





1 一 8 1 


2 
一 一 三 一 一 一 一， 了 1 三 巡 0.9.12b 
5 于 了 十 ] X2( ) ( ) 


X2(03) 一 


which conforms with Eq. (6.5.10)， 
The MAILAB program“nm651_2.m”uses a Symbolic computation routine 
“ilaplace()”to get the inverse Laplace transform，uses“eval()”to evaluate 
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Figure 6.5 Numericalyanalytical solutions of the continuous-time state equation (6.5.2)/(6.5.3). 


it, and plots the result as depicted in Fig. 6.3, which Supports this derivation pro- 
cedure. Additionally, it uses another Symbolic computation routine“dsolve()” 
to get the analytical solution directly. 


>>nm651_2 
Solution of Differential Equation based on LapJlace transfornm 
xs =[1s+1/s 人 (sr+1)*(-1+ 1 人 s) 
[ 1/(sS+1)*(-1+1/s) 1] 
xt =[ -1+tr+2x*exp(-t) ] 
[ -2*exp(-t) +1 


Analytical solution 
xt1 = -1+ 七 +2x*exp(-t) 
Xxt2 = -2xexp(-t) + 1 


%nm651 2: Analytical Solution for state eq. X'(t) = Ax(t) + Bu(t)(6.5.3) 
ClLear 
Syms S t %qdqeclare St as Symbolic variables 
A= [01;0 -1];B= [0 1]';s%Eq,.(6.5.3) 
x0= [1 -1] '; %initial value 
disp('`Solution of Differential Eq based on Laplace transform ') 
disp(`Laplace transformed Solution X(S) ) 
XSs = (S*eye(Size(A)) - A)^-1*(x0 + B/S) 5%5Eq.(6.5.5) 
disp('`Inverse Laplace transformed Solution x(t) ) 
xt = Ilaplace(Xs) %inverse Laplace transform %5Eq.(6.5.12) 
t0=0ojitf=2jN=45; s%initial/final tme 

to + [0O:N] *(tf - to)/N; %time Vector 
Xxtt = eval(xt:); %evaluate the inverse Laplace transform 
plLot(t,Xxtt) 
disp('Analytical Solution ') 
xt = dsolve('Dx1 = X2，Dx2 = -Xx2 + 1 ，'Xx1(0) = 1，Xx2(0) = -1 ); 
Xxt1 = Xxt.X1，Xxt2 = Xxt.Xx2 5%5Eq.(6.5.10) 
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6.5.2 Discretization of LTI State Equation 


In this section，we consider a discretization method of converting a continuous- 
time LII (linear time-invariant) state equation 


X'() 一 4X(C) 十 Bx() with the initial state X(0) and the input wx() (6.5.13) 
into an equivalent discrete-time LII state equation with the Sampling period 了 
X[72 十 ]] 三 4ox[z] 十 Bozx[P] (0.3.14) 
with the initial state X[0] and the input [2] 一 W(OT) for7p7 < 上 < (2 十 JJ 


which can be solved easily by an lterative Scheme mobilizing just Simple multi- 
plications and additions， 

For this purpose，we ITewrite the solution (6.3.8) of the continuous-time LII 
state equation with the initial time 1 as 


XI) 一 办 (一 四)X(0) + 4 一 T)BxT)dT (0.5.19) 


Under the assumption that the input is constant as the initial value within each 
Sampling interval 一 that is, [2] 一 WO2T) forn7 < < (2 十 JJ)7 一 wesubstitute 
加 一 PT andt 王 (2 十 DJ)7intothis equation to write the discrete-time LII state 
equation as 


(2 十 ])T 
X((2 十 1)7) =4xoD+ |/ 0((2 十 ])7 一 z)BxO2T)dTr 
用 了 


X[72 十 ]] 王 由 (T)X[7] 十 | (7 十 工 一 z)drBxI2] 
X[7 十 车 一 Axim] 十 Bozu[7] (0.5.10) 
Where the discretized System matrices are 
44 = 一 9 一 e (6.5.17a) 


人 G 一 77 十 了 一 7 
= 0(17 十 了 一 rz)drB 一 -| 4o)dos= 上 0(r)drB 
用 


(0.5.17b) 

Here, let us conslider another way of computing these System matrices，which 

is to the taste of digital computers. It comes from making use of the definition 
of a matrix exponential function in Eq. (6.5.60) to rewrite Eq. (6.5$.17) as 





Oo Oo 
了 了 
47 二 
4 一 e 二 全 1 2 本 (0.53.18a) 


Oo 


了 了 co 几 下 T 下 ATmA+1 
有 7 一 zr)dr 一 一 -drB = 一 一 一 有 一 炎 7B (0.53.18b 
4 人 yo /2 | ( ) 


10 一 0 
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where 
C9 下 半 
下 一 一 一 一 一 
2 
人 forNy>1l 
尝 Eee 让 一， 二 Se OF > 
2 3 N 一 1 AN 


(0.5.19) 
Now，we apply these discretization formulas for the continuous-time state 


equation (0.3.3) 
xi |0 1 XI1(1) 0 
ORDER DC 
with 1 一 | andz 人 =1YVf>0 


to get the discretized System matrices and the discretized state equation as 


| 
人 二 (659) | 1 1 工 一 e 
bg0) 一 ZHI-4= 工 量 让 主 村 | 


(0.3.20al) 
(65.179) CT) (6.5.209) 民 1 | (6.5.20b) 
0 e 
已， (6.5.17b) 上 本 
(6.3.209) | 1 | L 加 三 | | (6.5.20c) 
xz 十 1] 《 莹 9 4vx[] 十 Bou[m] 


X1[7 十] 1 1=-e 王 Te] 了 一 1 十 e 王 

| 请 = [| 和 | 而 和 52 
We don't need any Special algorithm other than an lterative Scheme to Solve 
this discrete-time state equation. The formulas (6.5.18a,b) for computing the 
discretized System matrices are cast into the Toutine“c2d_steq()” The Pro- 
gram“nm652.m” discretizes the continuous-time state equation (6.5.3) by using 
the routine and alternatively, the MAILAB builtrin routine“c2d()”. It solves 
the discretized state equation and plots the results as im Fig. 6.0. As long as 
the assumption that x[2] =W(OT) forpm7 < 上 < (十 ID)7isvalid, the solution 
(x[z]) of the discretized state equation 1S expected to match that (x(D)) of 
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Figure 6.6 The solution of the discretized state equation (6.5.21). 


the_ continuous-time state equation at every Sampling instant 上 一 7 and also 
becomes closer to x(f) Yi as the Sampling interval 7 gets Shorter (See Fig. 0.0)， 


s%nm652 .m 
%5 discretize a state eqn x'(t) = Ax(t) + Bu(t) to xn+ 1] = Ad*x[n] + Bdxurnl] 
Clear，CLf 
A= [01;0-1];B= [0;1]; s%Eq.(6.5.3) 
x0= [1 -1]; to0=0i tf= 2; 5%initial value and time span 
T=0.2; %sSampling interval(period ) 
eT = exp(-T); 
AD= [11-eTi0eTl%sdiscretized System matrices obtained analytically 
BD= [T+eT -1;1-eT] s%Eq.(6.5.21) 
[Ad,Bd] = c2d_steq(A,B,T,100) %continuous-to-discrete conversion 
[Ad1,Bd1] = c2d(A,B,T) %by the built-in routine 
t(1) = 0; xd(1,:) = x0; 5%initial time and initial Value 
fork=1:(tf -to)/T s%5solve the discretized state equation 
t(k+ 1) = kr*T; xd(k+ 1,，:) = xd(k,:)*Ad' + Bd' 
end 
Stairs([0; t'],[x0; xd])，hold on s%5stairstep graph 
N=100;t=to+ [Oo:N]'*(tf - to)/N; s%time (Column) vector 
x(:,，1) = t-1 + 2*exp(-t);j s#%5analytical Solution 
Xx(:,2) = 1-2*exp(-t); %Eq.(6.5-12) 
plot(t,X) 


function [Ad,Bd] = c2d_steq(A,B,T;N) 


if nargin < 4，N = 100; end 
I = eye(Size(A,2)); PSI = 

form=N:-1:1，PSI = 工 + Ar*PSI*T/(m+ 1); end s%5Eq.(6.5.19) 
Ad = 工 +ArPSI*T; Bd = PSI*TxB; %Eq.(6.5.18) 





6.5.3 High-Order Differential Equation to State Equation 


Suppose we are given an Nth-order scalar differential equation together with the 
initial values of the variable and its derivatives of up to order N 一 1，which is 
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called an IVP (mitial Value Problemy): 


[IIVP]Nw : xD = FoxzD xD ,xD ,xzw-D(CD) 


2 CN_D (0.5.22) 
With the initial values x(10) 一 X10,X (1 和) 一 X20，... ,2X (1 和) 一 XN0 
Defining the state vector and the initial state as 
0 X10 
和 2 一 X20 
二 开罗 
X() 一 | 273 二 《 X(10) 一 | 30 (0.5.23) 
XN 一 X-D XNO 


we can TIewrite Eq. (0.3.22) in the form of a first-order vector differential 
equation 一 that ls$, a state equation -一 as 


| 2 人 1) 
2X2(1) X3 (1 


x) | = x4(1) 
XN() GTX TCOCD | 
X (1) 一 fG, xX(D) with X(1o) 一 X0 (0.5.24) 
FEor example，we can convert a third-order scalar differential equation 
XG)(D 十 coxzO(D) 十 ax 十 aox(D 一 2 


into a state equation of the form 


2X1(D) 0 1 0 XI1 (四 0 
xD0|=| 0 0 10|+lolaoD (6525a 
2X3 (1 一 00 一 41 一 02 X3(1) ] 
X1(1) 
xD=[1 0 0]| ze2O) (6.5.25b) 
X3(1) 


6.5.4 Stiff Equation 


Suppose that we are given a vector differential equation involving more than one 
dependent variable with respect to the independent variable !. Ithe magnitudes 
of the derivatives of the dependent variables with respect to 1 (corresponding 
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to their changing rates) are Significantly different, such a differential equation ils 
Said to be stiff because it ls difficult to be solved numerically. For such a sti 仁 
differential equation， we Should be very careful in choosing the step-size in order 
to avolid numerical instability problem and get a reasonably accurate Solution 
within a Teasonable computation time. Why? Because we Should use a_ Small 
Step-Slze to grasp rapidly changing variables, and it requires a lot of computation 
to cover Slowly changing variables for such a long time as it lasts. 

Actually, there is no clear distinction between stiff and non-stiff differential 
equations, Since stiffness of a differential equation ls a matter of degree. Then, ls 
there any way to estimate the degree of stifftness for a given differential equation? 
The answer ls yes, if the differential equation can be arranged into an LII state 
edquation like Eq. (6.5.4), the solution of which consists of components having 
the time constants (modes) equal to the eigenvalues of the System matrix 4. For 
example, the System matrix of Eq. (6.3.3) has the eligenvalues 


改天 二 过 | 二 放 dcd||; ,||=se+D=o 3 一 0andsy 王 一 1 


Which can be observed as the time constants of two terms 1 = e0 and ein 


the solution (6.5.12). In this context, a measure of stiffness ls the ratio of the 
maximum over the minimum among the absolute values of Cegative) Teal parts 
of the eligenvalues of the System matrix 4: 


(4 = Max{|Re(Xi) (6.5.26) 
"ntRegl 关 可 是 


This can be thought of as the degree of unbalance between the fast mode and 
the Slow mode. 

Now,，what we mnust know is how to handle stiff differential equations. For- 
tunately， MAILAB has several built-in routines like“ode15s()”“ode23s()”， 
“ode23t()”, and“ode23tb ()”， which are fabricated to deal with stiff differen- 
tial equations effticiently. One may use the help command to see their detailed 
Usages. Let s apply them for a Van der Pol equation 


党 
一 学 -po0-PO)22+yO=0 过 yO=222 =-0 
[ Qi1 
(0.39.27a) 





which can be written in the form of a state equation as 


xf(D) iD) 二 下 考 克 裤 1 这 
/ 三 with 全 0.3.27b 
区 | 区 一 好 O)a0 一 aO) 动 和 0 
For this job, we defined this equation in an Mr-file named “df_van.m” and made 
the MAILAB program“nm654.m”"，where we declared the parameter 从 (mu) as 
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(c) Result obtained by ode45() (d) Results obtained by ode23( ), ode23s( )， 


ode23t( ), ode23tb( ) and ode15s() 


Figure 6.7 ”Numerical solutions of Van der Pol equation obtained by various routines,. 


a global variable so that it could be passed on to any related routines/functions 
as well as “df_van.m”. In the beginning of the program，we Set the global 
Parameter 人 to 25 and applied“ode_Ham()” with the number of Segments 
N=8700 and 9000. The results are depicted in Figs. 6.7a and 0.7b，which 
Show how crucial the choice of step-size js for a stiff equation， Next，we 
applied“ode45()”to obtain the solution depicted in Fig. 6.7c，which is almost 
the Same as Fig. 6.7b，but with the _ computation ttme less than one fourth 
of that taken by“ode_Ham()”. This reveals the merit of the MAILAB built- 
in routines that may Save the computation time as well as Spare ouUI trouble 
to_ choose the Step-Slize，because the Step-Size ls adaptively determined inslide 
the routines. Then，setting 从 = 200， we applied the MAILAB builtrin routines 
“ode45() /oode23() /oode15s() rode23s() /oode23t() /0ode23tb() ”to get 
the results that are little different as depicted in Fig. 6.7d，each taking the 
computation time as 


time = 24.9530 14.9690 0.1880 0.2650 0.2500 0.2820 


The _ computation time-efficiency of “ode15s() /oode23s() /0ode23t()2/ 
“ode23tb()”(designed deliberately for handling stiff differential equations) oveT 
“ode45() /oode23()”becomes prominent as the value of parameter 人 (mnu) gets 
large, reflecting high stifftness. 
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%nm654 .nm 

% to Solve a Stiff differential eqn called Van der Pol equation 
global mu 

mu=25，to=0;i tf = 100; tspan = [to tf];j xo = [2 0]; 

[tH1,xH1] = ode_Ham( 'df_ van' ,tspan,x0,8700) ; 
SubpJlot(221)，plot(tH1,xH1) 

tic,[tH2,XxH2] = 0ode_Ham( ' df van' ,tspan,x0,9000); time_Ham = toc 
tic,[t45,Xx45] = 0de45('df van' ,tspan,Xx0); time 045 = tocC 
SubpJlot(222)，plot(tH2,xH2)，Ssubplot(223)，plot(t45,Xx45) 

mu = 200j tf = 200j tspan = [to tf]; 

tic,[t45,Xx45] = 0de45(' df van' ,tspan,x0); time(1) = tocij 
tic,[t23,x23] = 0de23('df van' ,tspan,x0); time(2) = tocij 
tic,[t15s,xl15s] = 0de15s('df van' tspan,x0); time(3) = toci 
tic,[t23s,x23s] = 0de23s('df van' ,tspan,x0); time(4) = toci 
tic,[t23t,x23t] = 0de23t(' df van' ,tspan,x0); time(5) = toci 
tic,[t23tb,x23tb] = ode23tb( df _van' ,tspan,x0); time(6) = toci; 
plot(t45,X45，t23,Xx23，t15s,X15SsS，t23s,Xx23Ss，t23t,X23t，t23tb,Xx23tb) 
disp(`” 0ode23  ode15s  ode23Ss 0de23t 0ode23tb ' ) 

廿 Ime 


function dx = df _ van(t,X) 

%Van der Pol differential equation (6.5.27) 

global mu 

dx=zeros(SIize(Xx) ); 

dx(1) = X(2); dx(2) = mu*x(1-X(1).^2).*x(2) - X(1); 





6.6 BOUNDARY VALUE PROBLEM (BVP) 


A boundary value problem (BVP) is an Nth-order differential equation with Some 
of the values of dependent variable x(f) and its derivative Specified at the initial 
time 加 and others Specified at the final time 17. 


[BVP]Nv : XICD = FU xD XXXW-DCD) 


with the boundary values x(0) 一 Xil0,x(D) 一 2 XCOT-DGUN) 一 XNN_l 
(6.6.1) 
In Some cases, Some relations between the initial values and the final values may 
be given as a Imixed-bpoundary condition instead of the initialfnal values Spec- 
ified. This section covers the Shooting method and the finite diffterence method 
that can be used to Solve a Second-order BVP as 


[BVP]， :xD = Fo xx CD) with X(10) = xzo,X(Gp) 一 ZF (6.6.2) 


6.6.1 Shooting Method 


The idea of this method is to assume the value of x' (fo), then solve the differential 
equation (IIVP) with the initial condition [x(1o) x“(to)] and keep adjusting the value 
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of x“ (1o) and solving the IVP repetitively until the final value x(tr) of the solution 
matches the given boundary value xr with enough accuracy. It is Similar to 
adjusting the angle of firing a cannon So that the shell will eventually hit the target 
and that's why this method is named the sjpootizg method. This can be viewed 
as a nonlinear equation problem, 让 we regard x“ (10) as an independent variable 
and the difference between the resulting final value x(tr) and the desired one xX7 
as a (mismatching) function of x“(1o). So the solution scheme can be Systemized 
by using the secant method (Section 4.9) and is cast into the MAILAB routine 
“bvp2_shoot ()”. 


(cf We might have to adjust the shooting position with the angle fxed, instead of adjust- 
jing the shooting angle with the position fxed or deal with the mixed-boundary 
conditions. See Problems 6.6, 6.7, and 6.8. 


FEor example, let's consider a BVP consisting of the Second-order differential 
equation 


xD) 一 2x2() 十 4f xDxz GD) with x(0) 二 本 X(]) 一 3 (6.6.3) 


function [t,Xx] = bvp2_shoot(f,to,tf,x0,xf,N,tol,kmax) 
%5T0O SoOlVve BVP2: [xl1,x2]' = f(t,Xx1,Xx2) with xl(t0) = X0，Xx1(tf) = Xf 
if nargin < 8，kmax = 10; end 
if nargin < 7，tol = 1e-8;j end 
if nargin < 6 N = 100; end 
dxo(1) = (xf - x0)/(tf-t0); 55 the initial guess of Xx' (t0) 
[t,x] = ode_RK4(f,[to tf],[xo dxo(1)],N); s% start Up with RK4 
plot(t,x(:，, 1))，hold on 
e(1) = Xx(end,1) - xf;j%X(tf) - xf: the 1st mismatching (deviation) 
dx0(2) = dx0(1) - 0.1*sign(e(1)); 
for kK = 2: kmax-1 
[t,x] = 0ode_RK4(f,[to tf],[xo dxo(k)],N); 
plot(t,x(:，1)) 
#%difference between the resulting final value and the target one 
el(k) = Xx(end,1) - xf;i % X(tf)- Xf 
ddx = dxo(k) - dxolk - 1); 点 difference between Successive derivatives 
if abs(e(k))< tol | abs(ddx)< to1l，breakj end 
deddx = (e(k) - e 人 k - 1))/ddxi 55 the gradient of mismatching error 
dxoO(k + 1) = dx0(k) - e(k)/deddxji %move by secant method 


%5do_shoot to Solve BVP2 by the Shooting method 

t0=0Oltf= 1;) x= 1/4; xf = 1/3; %initial/final times and positions 
100; tol = 1e-8; kmax = 10; 

[t,x] = bvp2_shoot( 'df661' ,to,tf,x0,xf,N,tol,kmax) ; 

xo=1./(4- tx*t)jerr = norm(x(:，1) - xo)/(N+ 1) 

plot(t,x(:,，1)，b'，t,Xxo rr' ) %compare With true Solution (6.6.4) 


function dx = df661(t,Xx) %5Eq,.(6.6.5) 
dx(1) = X(2))j dx(2) = (2*Xx(1) + 4*xtxXx(2))*xx(1); 
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The solution x(D) and its derivative X' (1) are known as 


1 21 2 
X(1) 一 不 二 7 and  X (四 一 人 4 二 7 一 21X (T) (0.0.4) 
Note that this Second-order differential equation can be written in the form of 
state equation as 


xD |] 22(D) won [xDO1_ [mo=L4 
xD] [2xt0) 十 4 x1(D)xz2() xD | |xr=1/3 
(0.0.5) 
In order to apply the Shooting method，we set the initial guess of xz(0) 一 


X'(0) to 
XF 一 X0 
dx0[1] = xz (0) = 一 一 一 (0.0.0) 
1r 一 加 


and Solve the state equation with the initial condition [xl1(0) xz2(0) = dx0O[1]]. 
Then, depending on the sign of the difference e(1) between the final value xi1(]) 
of the Solution and the target final value YXr，we make the next guess dxX0[2] 
largersmajller than the initial guess dx0O[1] and solve the state equation again 
with the initial condition [xl1(0) dx0[2]]. We can start up the secant method 
with the two initial values dx0O[1] and Zx0[2] and repeat the iteration until the 
difference (error) e(k) becomes sufficiently Small. For this job，we compose 
the MAILAB program“do_shoot,m"，which uses the routine“bvp2_shoot () 7” 
to_ get the numerical solution and compares it with the true analytical Solution. 
Figure 0.8 Shows that the numerical solution gets closer to the true analytical 
Solution after each round of adjustment. 


(Q) Why don't we use the Newton method (Section 4.4)2 

(A) Because, in order to use the Newton method in the shooting method, we need IVP 
Solutions instead of function evaluations to find the numerical Jacobian at every 
iteration，which will require much longer computation time. 
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Figure 6.8 The solution of a BVP obtained by using the shooting method. 
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6.6.2 Finite Difference Method 


The idea of this method is to divide the wphole interval [fo, 好 ] into N segments 
of width 瑚 三 (一 加) 人/N and approximate the first 久 second derivatives in the 
differential equations for each grid point by the central difference formulas. This 
leads to atridiagonal System of equations with respect to (CN 一 ]J) variables {2xi 一 
X( 加 十 11)1 一 1...,N 一 1.However in order for this System of equations to 
be solved easily, lt should be linear, Implying that its coefficients may not contain 
any term of x. 

For example, let's conslider a BVP consisting of the second-order linear dif- 
ferential equation 


X (ID) 十 G1G)X (CD 十 ao(D)xX(GD 一 2G) with x(o) 一 Xo,x(Gr) 一 ZX (6.6.7) 


According to the finite difference method，we divide the solution interval 
[io,tr] into N segments and convert the differential equation for each grid point 
右 一 四 十 17 into adifference equation as 


Xi+1 一 2Xi 十 Xi j 上 +1 一 和 -1 
一 十 Go0iX 三 民 
(2 一 pan)xzi-1 十 (4 十 212ao)xi 十 (2 十 pa)Xirl 二 21720065 (0.0.8) 


Then, taking account of the boundary condition that xo 一 X(1o) and xx 一 X(Lr)， 
we collect all of the (CN 一 IJ equations to construct a tridiagonal System of 
equations as 


一 4 十 2712aoi 2 手 jall 0 @ 0 0 0 
2 一 ja -4 二 212ao 2+hpa 。 0 0 0 
0 2 一 1al13 一 4 本 2712a03 @ 0 0 0 
外 外 国 国 外 外 国 
0 0 0 e 一 人 后 212ao N_3 2 富 jal N_3 0 
0 0 0 。 2--jhalNw > -4+212aow > 2+7jPaliN > 
0 0 0 @ 0 2 一 PalN 1l 一 4 十 212ao.Nw_1 
X1 21271 一 (2 一 Paii)xo 
X2 27212 
2 272103 
光 间 -入 | 衬 1 (6.6.9) 
XN-3 212xN_3 
XN-2 212xN-_2 
XN-1 2Hp2uN_1 一 (人 2 一 AnalNw_i)xN 


This can be solved efficiently by using the MAILAB routine“trid()”，which 
is dedicated to a tridiagonal System of linear equations， 
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The whole procedure of the finite difference method for solving a second-order 
linear differential equation with boundary conditions is cast into the MAILAB 
Ioutine“bvp2_fdf()”. This routine is designed to accept the two coefficients 
al and ao and the right-hand-side input & of Eq. (6.6.7) as its first three input 
arguments, where any of those three Input arguments can be given as the function 
name in case the _ corresponding term is not a numeric value，but a function 
of tme 1!. We make the program“do_fdf”to use this routine for Solving the 
Second-order BVP 


二 0) 一 3 一 0 with x(1) 一 3,x(C) 王 3 (6.6.10) 


function [t,x] = bvp2 fdf(a1l,a0,utotf,xo,xf,N) 
5 SO]JVe BVP2: X" + alx*x' + ao0*x = U With x(t0) = X0，X(tf) = Xf 
5 by the finite difference method 
h= (tf- to)/Ni h2 = 2x*hxhi 
在 = to+[0:N] *hi 
If ~ISsnumeric(a1l)， = al(t(2:N)); %if al name of a function of 七 
elseif length(al ) 1，al = alxoneSs(N - 1，,1); 
end 
If ~Isnumeric(a0) ， a0(t(2:N)); %if a0 = name of a function of 七 
elSseif length(a0) 1，a0 = a0*ones(N - 1,1); 
end 
if ~ISsnumeric(u)，U= ut(2:N)); %ifu= name of a function of 芋 
elSseif length(u) == 1，U = UxoneSs(N-1,1); 
elSeu=U(:); 
end 
A= zeros(N - 1IN - 1); b 
ha = hx*al(1); A(1,1:2) = [ 
b(1) = b(1)+(ha - 2)*x0i 
form= 2:N - 2%Eq.(6.6.9) 
ha = hxal(m); Am'm- 1:m+ 1) = [2-ha -4 + h2xa0(m) 2+ hal]; 
end 
ha = hx*al(N - 1)}) AN- 1IN- 2':N- 1) = [2 - ha -4+ h2x*ao0N - 1)]; 
b(N - 1) = b(N-1)-(ha+2)*xf; 
X= [xo trid(A,b)” xf] ; 


= h2xu; 
-4 + h2*a0(1) 2 + hal]; 


function x = trid(A,b) 

s%5 SolVve tridiagonal System of equations 

N = Size(A,2) 

form= 2:N5% Upper Triangularization 
tmp = Almm- 1)/Am- 1m- 1); 
Amm) = A(m;,m) -Am - 1,m)*tmp; Amm - 1) 
b(m,:) =bm:) -bm - 1,:)*tmpy; 

end 

x(N,:) = b(N,:)/A(NN); 

form=N- 1: -1:15%Back Substitution 
X(m;:) = (blm,:) -Amsm + 1)*xm+ 1))7ACmsm); 

end 
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#%qdo_ fdf to Solve BVP2 by the finite difference method 

Clear，CLf 

to0=1;Xx=5;) tf=2;) xf= 3jN= 100; 

al = inline('2./t't');a0 = inline('-2./t./t't');)uU= 0; %Eq.(6.6.10) 
[tt,x] = bvp2 fdf(al,a0,utotf,xo,xf,N) ; 

%use the MATLAB built-in command '“bvp4c() 

df = inline('[x(2); 2./t.*(xX(1)./t - X(2))] ，t' ，Xx'); 

fbc = inline('[x0(1) - 5;) xf(1) - 3]'，x0'，Xxf'); 

Solinit = bvpinit(1Linspace(to,tf,5),[1 10]); s%initial SolLlution interval 
Sol = bvp4c(df,fbpc,solinit,bvpset('` RelTol ,1e-4) ); 

X_bvp = deval(Ssol,tt); xbv = X_bvp(1,，:) ; 

#%use the Symbolic computation command '` qdqsolve() 

Xxo = dsolve('D2x + 2*(Dx - X/t)/t=0 ，x(1) =5，Xx(2) = 3') 

Xxot = SubSs(Xx0o，t' tt); %xot=4./tt./tt +tt; %true analytical solution 

err fd = norm(Xx - xot)/(N+1) %error between numerical/analytical Solution 
err _bvp = norm(xbv - xot)/(N + 1) 

plot(tt,Xx，b' tt,xbv，r' ytt,xot，k') %scompare With analytical Solution 





We run lt to get the result depicted in Fig. 6.9 and，additionally，use the 
Symbolic computation command “dsolve()”and“subs()”to get the analytical 
Solution 


4 
X(f) 一 夺 十 元 (0.0.11) 


and Substitute the time vector into the analytical solution to obtain its numeric 
values for check. 

Note the following things about the Shooting method and the finite differ- 
ence method: 


。 While the Shooting method is applicable to linearnonlinear BVPs, the finite 
difference method is suitable for linear BVPSs. However， we can also apply 
the finite difference method in an iterative manner to Solve nonlinear BVPs 
(See Problem 0.10). 














| 
1 1.2 1.4 1.6 1.8 上 2 


3 1 





Figure 6.9 Asolution of a BVP obtained by using the finite difference method. 
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。 Both methods can be modified to solve BVPs with mixed-boundary condi- 
tions (See Problems 6.7 and 6.8). 

。 JIn MAILILAB 6.x，the “bvp4c()” command is available for solv- 
ing linearnonlinear BVPSs with mixed-boundary conditions (See Prob- 
lems 6.7--0.10). 

。 The Symbolic_ computation command“dsolve()”introduced in Section 
0.3.1 can be used to solve a BVP so long as the differential equation is lin- 
ear, that is, its coefficients may depend on time 1, but not on the (unknown) 
dependent variable x(1). 

。 The usages of “bvp4c()”and“dsolve()”are illustrated in the program 
“do_fdf”, where another Symbolic computation command “subs()”is used 
to evaluate a Symbolic expresSlion at certain value(S) of the variable. 


PROBLEMS 


0.0 MAILAB Commands quiver() and quiver3() and Differential Equation 
(a) Usage of quiver() 

Type help quiver” into the MAILAB command window, and then you 
will see the following program Showing you how to use the quiver() 
command for plotting gradient vectors. You can also get Fig. P6.0.1 by 
running the block of statements in the box below. Try it and note that 
the Slze of the gradient vector at each point ls proportional to the Slope at 
the point， 


5%5do_quiver 
[x,y] = meshgrid(-2:.5:2,-1:.25:1); 
Z = X.*exp(-xX.^2 - y.^2); 
[px;,py] = gradient(zZ,，.5,，.25) ; 
contour(x,y,Zz)，hold on，quiver(x,y;，px;,py) 
axiSs image %the Same as AXIS EQUAL except that 
%the plot box fits tightly around the data 
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Figure P6.0.1 Graphs obtained by using gradient(), contour(),quiver(). 
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Figure P6.0.2 ”Graphs obtained by using surfnorm(),quiver3(),，Surf(). 


(b) Usage of quiver31() 


(C 


S 旋 


You can obtain Fig. P6.0.2 by running the block of statements that you 
see after typing “help quiver3” into the MAILAB command win- 
dow. Note that the“surfnorm()”command generates normal vectors 
at points specified by (x, y, Z) on the surface drawn by “surf()”and 
the“quiver3()”command plots the normal vectors. 


%do_quiver3 

Clear，Clf 

[x,y] = meshgrid(-2:.5:2,-1:.25:1); 
Z=X.xexp(-x.^2 - y.^2); 


Surf(x,y,Z)，hold on 
[uv,w] = Surfnorm(x,y,Z) ; 
quiver3(x,y,， ZuUVW) 





Gradient Vectors and One-Variable Differential Equation 

We might get the meaning of the solution of a differential equation 
by using the“quiver()”command，which is used in the following 
program “do_ode.m” for drawing the time derivatives at grid points as 
defined by the differential equation 


2 一 一 >( 人 十 1 with the initial condition y(0) 二 0  (P6.0.1) 


The Slope/direction field together with the numerical Solution in 
Fig. P6.0.3a is obtained by running the program and it can be ITegarded 
as a Set of possible solution curve segments. Starting from the initial 
point and moving along the Slope vectors，you can get the solution 
curve， Modify the program and run it to plot the Slope/direction 
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SEE] 1 一 一 一 一 > 一 一 一 一 ] 
和 2 和 到 -也 
8hF? 了 到 到 到 一 2 | 一 
OA :2000 | 世 二 
| 区 ] 
| > 过 轧 和 iii ol 人 人 
23223332 
0 2 ss ] 
[区 省 和 拘 奉 帮 万 大 万 洛 入 

0 、 
0 0.5 1 1.5 2 避 

(a) The graph of ay vs. dtfor xb= yo 

y(bD=-yYb+1i (b) The graph of ax2 vs. ax for 必 _ -2 半 


and its solution for the initial condition 


and its solution for WXO) = 0 
[x4(0) x2(O0]=[I1-1or[1.5-0.5] 


Figure P6.0.3 ”Possible solutions of differential equation and slope/direction field. 


field (xz(1) versus X1()) and the numerical solution for the following 
differential edquation as depicted in Fig. P6.0.3b. 


X1(1) 一 X2(1) . xi(OD0)] T『1 1 5 
也 0 汪 二 二 玫 | 一 加 区 (6.0.2) 


do_ode .m 
This uses quiver() to plot possible solution curve Segments 


called the Slope/directional field for y'(t) +y=1 


=0ijtf=2; tspan= [to tflj xo = 0 
[t,y] = meshgrid(to:(tf - to)/10:tf;,0:.1:1); 


pt = ones(Size(t)); py= (1 - y).*xptijs%qdy= (1 - y)dt 
quiver(t,yypt,py) %5y(displacement) vSs. t(time) 

axis([to tf + .2 0 1.05])，hold on 

dy=inline('-y+ 1 ，t ，y' ); 

[tR,yR] = 0ode_RK4(dy,tspan,x0,40) ; 

for k = 1:length(tR)，plot(tR(k),yR(k)，rx' )，pause(0.001); end 





0.1 A System of Linear Time-Invariant Differential Equations: An LII State 
Equation 
Consider the following state equation: 
Xi | 0 1 | | xD) 0 . Xl(0O) | |1 
区 攻 攻 二 | 二 二 中 汪 | 光 | 全 全 | 二 | 二 | 
(P6.1.1) 


(a) Check the procedure and the result of obtaining the analytical Solution 
by using the Laplace transform technique. 


296 ORDINARY DIFFERENTIAL EQUATIONS 


X(G) 一 [S1 一 4]-Ix(O) 二 BUG)) 


| 中 
SG+3) 十 2 一 2 3 0 0 | ys 
1 | 

GC+HDGC+2)L -2 二 1 


(2 十 35 十 1D)/G 十 DG 十 2) 
-1/G+lDG 二 2) 








人 
一 一 一 全 基 一 一 本 .上 .3 
8 S+1T +2 2 2 5 

二 1 一 一 2 
X2(S) 一 yYTT 十 了 2 X2(1) 一 一刀 十 人 (P6.1.2b) 


(b) Find the numerical solution of the above State equation by using the 
routine“ode_RK4()”(with the number of Segments N = 90) and the 
MAILAB built-in routine“ode45()”. Compare their execution time 
(by using tic and toc) and closeness to the analytical Solution. 


6.2 A Second-Order Linear Time-Invariant Differential Equation 


Consider the following second-order differential equation 
X" (1) 十 3x (0 十 2x(0) 一 1 with x(0) 一 1x 0) =0 (P6.2.1) 


(a) Check the procedure and the result of obtaining the analytical solution 
by using the Laplace transform technique. 


S2X(G) 一 x(0) 一 sx(0) 十 3G6XCG) 一 x(0)) 十 2X(G) 一 - 


5 十 3 十 1 1 ] 
X( 二 ss 四 一 二 十 e 开 一 二 ee (P6.2.2 
二 人 
(b) Deftine the differential equation (P6.2.1) in an M-file so that it can be 
passed to the MAILAB routines like“ode_RK4()”or“ode45()”as 
their input argument (See Section 0.3.1). 


6.3 Ordinary Differential Equation and State Equation 


(a) Van der Pol Equation 
Consider a nonlinear differential equation 
d 2 Q 
本 27) 一 (一 y CO) 元 7() 十 y( 人 =0 with 从 =2  (P6.3.1) 
Compose a program to solve this equation with the initial condition 
[y(0) y (0)] = [0.5$ 0] and [一 1 2] for the time interval [0, 20] and plot 
y“(D) versus y() as well as y() and y(D along the 1-axis. 
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(b) Lorenz Equation: Turbulent Flow and Chaos 


(Cc 


(d 


(e 


) 


AN 


ea 


Consider a nonlinear state equation . 


X1(0) 一 GO) 一 2X1(D) 和 =10 
2 一 (二 和 一 xD)xzD 一 2 人 with 入 =20~100 (P6.3.2) 
X3(1) 一 X1(D)X2(1) 一 YX3(1) /三 2 


Compose a program to solve this equation with 和 = 20 and 100 for the 
time interval [0,10] and plot x3 (1) versus X1(1). Let the initial condition 
be [xl1(0) xz2(0) xz3(0)] = [一 8 一 16 80]. 

Chemical Reactor 

Consider a nonlinear State equation describing the concentrations of 
two reactants and one product in the chemical process. 


X1() 一 QCL 一 2X1(0D) 一 PXICD)X2(D) 4 一 
X2(1) 一 0(l2 一 X2(1)) 一 pxi()Xz2(1) with 一 2 (P06.3.3) 
X3(1) 一 一 0X3() 十 DXI(CD)X2 (ID 2U1 一 3,zx2 一 9 


Compose a program to Solve this equation for the tme interval 
[0，H] and plot xl1(0，x2(1)，and x3(1 和 Let the initial condition be 
[xzl(0) xz2(0) xz3(0)] = [1 2 3]. 

Cantilever Beam: A _ Differential Equation w.I.t a Spatial Variable 
Consider a nonlinear state equation describing the vertical deflection of 
a beam due to its own weight 


d2 dy ZN 7 
-了 = pog ( 十 他 | (4 加 3 于 和 | (P6.3.4) 


where ,天 = 2000 kg .m3/s2，p = 10 kg/m，8g = 9.8 ms , 工 = 2 m. 
Write a program to Solve this equation for the interval [0, 志 ] and plot 
y(0). Let the initial condition be [y(0) yY(0)] = [0 0]. Note that the 
Physical meaning of the independent variable for which we usually use 
the Symbol 浊 in writing the differential function ls not a time，but 
the X-coordinate of the cantilever beam along the horizontal axis im 
this problem. 

Phase-Locked Loop (PLD) 

Consider a nonlinear state equation describing the behavior of a PLL 
circuit depicted in Fig. P6.3.1. 


加 a 二 1500 
iD) 一 0 
xD 一 sin(wo 四 
X2(1) 一 XI1(G) 十 ac 


y(D) 一 X1(C0D 十 we (P6.3.5b) 
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直 


(9) 


loop filter 
LUD = sin(wob 下 |] Xi 


将 
” 1+T7S "<D) > 


- MX2( 人 | 1 
oscillator | - 






































cos(x ( 切 ) 


Figure P6.3.1 The block diagram of PLL circuit. 








where wo = 王 2100r [rad/s] and mowe = 2000r [rad/s]. Compose a Pro- 
gram to solve this equation for the time interval [0,0.03] and plot >y(D) 
and wo. Let the initial condition be [xl1(0) xz(0)] = [0 0]. Is the output 
y(D) tracking the frequency mwo of the input x(D2 

DC Motor 

Consider a linear differential equation describing the behavior of a DC 
Imotor System (Fig. P0.3.2) 


d20(00) d0() 











JJ 十 瑟 =TO) = Ri 
妈 了 (p6.3.6) 
人 
Qi Qi 


Convert this System of equations into a first-order vector differential 
equation 一 that 1$，a state equation with respect to the state vector 
[CD) 9 (0 7GD)]. 

RC Circuit: A_ Stiff System 

Consider a two-mesh RC circuit depicted in Fig. P6.3.3. We can write 
the mesh equation with respect to the two mesh currents D( 人 ) and 
12(1) as 








己 上 
AAA 人 v mm 个 
已 
vV 人 (四 ) 十 angular 

















displacement 
忆 ) 























back e.m.f. vv 人力 =Koo( 人 =HKoo( 
tordque 7 (bb = K7/b) 


Figure P6.3.2 A DC motor system. 
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PR = 100[9] Ci =10[wF] 
| 


v(D = i[V] 已) 1 1 人 站 


Figure P6.3.3 Atwo-mesh RC circuit. 

















1 疙 
RID(CD 十 二 / ii(r)dr 十 RiO 一 P(OD) 一 VOD) 一: 


Ra2(2(1) 一 三 (1) 十 二 |/ ji2(r)dr 三 0 (P6.3.7al) 
2 v 一 co 


In order to convert this System of differential equations into a state 
equation， we differentiate both sides and rearrange them to get 














2 Ra2O ， du 
(Ri 十 R2) 二 本 ) 一 记 一 1 
d d 
一 PR， mg PR 20 ，， + 二 PO= =0 (P6.3.7b) 


有 II 十 R> 人 人 玉 ， (D) 加 1 一 立 (D)/C 
| 一 人 2 R2 | 四 至 | 一 DPCD/C2 | (P6.3.7c) 


_1 
1 站 | | RI+R 一 R2 1 一 六 (DC 
区 加 | 一 R2 R2 | | 一 2(D/C2 | 人 


芭 医 。 人 一 GI/C2 | | 可 区 | 全 
-CIHCI 一 (CGI 十 GC2)/C2 | 120 CGI | 
(P6.3.7d) 
where xs (1) denotes the unit Step function whose value 18 1 (one) VY 
1 二 0. 

(iD After constructing an M-file function“df6p03g.m”which defines 
Eq. (P6.3.7d) with Ri = 100[S], Ci = 10[E, Ra2 = TIKES], C2 一 
10[E]，use the MAILAB builtin routines “ode45()” and 
“ode23s()”to Solve the state equation with the zero initial con- 
dition 7(0) 一 22(0) =0 and plot the numerical solution 2() for 
0 <! < 0.035 S.Forpossible change ofparameters, you may declare 
有 IC1, R2，C2 as global variables both in the function and in the 
main program named, say，nm6p03g.m”. Do you see any Symptom 
of stiffness from the results? 
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(过 开 we apply the Laplace transform technique to solve this equation 
with zero initial condition i(0) 王 0，we can get 


(8) | (6.5.5) -1 
了 4 三 [S7 一 4] BUS) 


攻 ， CGI/C， | 
Cl1 | y 


CI/C +(GI 二 GC2)/C2 
CI1 
AD F< 二 记 7S 交 于 全 RS <RRS 主 二 有 二 
5 十 (GUCCI 十 (CGI 十 GC2)/C2)s 十 GIG2/C1IC> 
1/100 
82 十 2100s 十 100000 
1/100 


由 


(s 十 2051.25)(s 十 48.75) 


7 1 1 
”200250 (于 -0 


2(D) 侍 人 上 区 (P6.3.7e) 





200250 


where 和 1 = 一 2051.23 and 和 ;> = 一 48.73 are actually the eigenval- 
ues of the System matrix 4 in Eq. (P6.3.7d). Find the measure of 
stiffness defined by Eq. (6.5.20). 

(iii) Using the MAILAB symbolic computation command “dsolve()”， 
fnd the analytical solution ofthe differential equation (P6.3.7b) and 
plot j2 (1) together with (P6.3.7e) for 0 < 上 < 0.03 S. Which of the 
two numerical solutions obtained in (GD ls better? You may refer to 
the following code: 


Syms R1 R2 C1 C2 

1 = dsolve('(R1+R2)*Di1 - R2*Di2 + il1/C1 = 1 ，... 

" -R2*Di1 + R2*Di2 + iI2/C2' ，i1(0) = 0'，i2(0) = 0'); 5%(P6.3.7b) 
100; R2 = 1000; Cc1 = 1e-5; C2 = 1e-5; 

0j tf=0.05;j 不 = to+(tf-t0)/1100*[0:100]; 

i2t = eval(1i.i2); plot(t,i2t，m') 





606.4 Physical Meaning of a Solution for Differential Equation and Its Animation 


Suppose we are going to Simulate how a vehicle vibrates when it moves 
with a constant Speed on a rugged way, as depicted in Fig. P6.4a. Based on 
Newton's Second law, the Situation ls modeled by the differential equation 
(P6.4.1). 


dl Q 
1 FID) 二 2 元 OO 一 2(D)) 十 天 (7 一 GOD) 一 0 (P6.4.1) 
with y(0) = 0，y (0) = 0 
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%5do_MBK 
C]l 
t0=0oitf= 10; x = [00]; 
[t1,x]l = ode_Ham( 'f_MBK' ,[tO tf],xo0) ; 
dt = tl(2) - tl1(1); 
forn= 1:lLength(t1) 
udu_MBK(t1(n) ) ; 
end 
figure(1)，clLf 
animation = 1; 
if animation 
figure(2)，clLf 
draw_MBK(5,1,x(1,2),uU(1)) 
axis([-2 2 -1 14])，axis(' equal ) 
pause 
forn=1:1Length(ti) 
ClLf，draw_MBK(5,1,x(n;2),uU(n)，b') 
axis([-2 2 -1 14])，axis(' equal ) 
pause(dt) 
figure(1) 
plot(t1(n),uUCn)，r. ，tin),xn,2)，b.，) 
axis([O tf -0.2 1.2])，hold on 
figure(2) 
end 
draw_MBK(5,1,x(n,2),uU(n) ) 
axis([-2 2 -1 14])，axis(' equal ) 
end 


function [udul = udu_MBK(t ) 
II= fix(t); 

if mod(i,2) == 0，U = ti du 
elseu=1 

end 


function draw_MBK(n,w,y，uUcolor) 
: the # of spring windings 
: the width of each object 
: displacement of the top of MBK 
: displacement of the bottom of MBK 
nargin < 5，color = k'; end 
=[-wu+4; p2=[-w 9+y]i 
= 0; ym= (p1(2) + p2(2))/12; 
xm + wxr1.2*[-1 -1 1 1 -1]) 
p2(2) + WwW*r[13 3 1 1]; 
plot(xM,yM,color)，hold on %Mass 
spring(n,p1,p2,w,color) %5Spring 
damper(xm + wp1(2),p2(2),wW;,color) %5Damper 
Wheel_my(xm,p1(2)- 3*wwWcolor) %Whee1 


function dx = f_MBK(t,xX) 
1;B=0.1;K= 0.1; 
[u,dul = udu_MBK(t) ; 
dx = xx[0 1; -B/M - K/M]'+[0O (Kxu + Bxdu)/VM]; 
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function Spring(n,p1,p2,w,color) 
s%draw a Spring of n windings，width w from p1 to p2 
if nargin < 5，color = kj; end 

(p2(1) - p1(1))/2; d = (p2(2 

(p2(1) + p1(1))/2; 9 = (p2(2 

-1:0.01:1; 不 = (Yy+1)*xpix(n 

-0.5*Wx*Sin(t); y = y+0.15*(1 

y(1); b=y(LIength(Xx) ) ; 

= 2*(Yy -al)/(b- a)-1; 

yyYS= dry - CrX+ gj XxS = X+f; XxSl= [f Tf]); 
yyS1 = yyS(Iength(yyS))+[0 w]; yyS2 = yyS(1)-[0 w]; 
plot(xXxS,yyS,color，XxxS1,yyS1,color，XxxS1,yyS2,color) 


function damper(xm,y1,y2,w,color) 
s%5draws a damper in (xm-0.5 xm + 0.5 y1 y2) 


if nargin < 5，color = 'k'; end 

ym = (yY1 + y2)/12; 

xD1 = xm+ WwWwr[0.3*[0 0 -1 = [y2+w ym ym ynm]i; 
XD2 xm + WwWx*r[0.5*[-1 -1 ， ym+wr[1 -1 -1 

1]; 

xD3 = xm+ [0 0]; yYD3 = [y1 ym]l - wWi 
plot(xD1,yD1,color，xD2,yD2,color，xD3,yD3,color) 


function wheel_my(xm,ym,w,color) 

s%5draws a Wheel of Size WwW at center (xm,ym) 

if nargin < 5，color = kj; end 

XxW1 = xm+ WwWri.2*r[-1 1];) yWI= ym+ WwWr[2 2]; 

XW2 = xmx*[1 1]; yW2 =ym+WwWwrr[2 0]; 
plot(xW1,yW1,color，XxW2,yW2,color) 

th = [0:100]/50*pi; plLot(xm + jxym+Wwxexp(j*th),， color) 
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(a) The block diagram (b) The graphs of the input wb and the output y 


Figure P6.4 A mass-spring-damper system. 
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where the values of the mass, the viscous friction coefficient, and the Spring 
constant are given as M = 1 kg, 有 =0.1NSmand 天 一 0.1 Nm,respec- 
tively. The input to this System is the movement V(I) of the wheel part 
causing the movement y(1) of the body as the output of the System and is 
approximated to a triangular wave of height 1 m, duration 1 Ss, and period 
2 8 3as depicted in Fig. P6.4b. After converting this equation into a state 
equation as 


[区 | -em -ar[eovoyuooeo 
2 人 | | -KM -B/M | zz (B/AM)ZCD 十 (K/AM)ZCD 


(P0.4.2) 
X1(0) 0 
with 攻 光 大 四 


We can Use Such routines as ode_Ham()，ode45(),... to solve this state 
equation and use Some graphic functions to draw not only the graphs of 
y() and x()，but also the animated Simulation diagram.， You can run 
the above MAILAB program “do_MBK.m”to see the results. Does the 
SuSpension System made of a Spring and a damper as depicted in Fig. 
P6.4a absorb effectively the Shock caused by the rolling wheel so that the 
amplitude of vehicle body oscillation is less than 1/S times that of wheel 
OoScillation 


(cf) I one is interested in graphic visualization with MAILAB, he/she can refer to 
[IN-1]. 


A Nonlinear Differential Equation for an Orbit of a Satellite 


Consider the problem of an orbit of a satellite,whose position and velocity 
are obtained as the solution of the following state equation: 


xl1(0) 一 X3(D) 

22(1) 一 X4() 

2(D = 一 GMEXIDVACTCOD 十 好 (0D)302 (P6.5.1) 
xD = 一 GMEz2(DVACTCD 十 好 (D)302 


where G = 6.672 x 10-1 N m2/kg” is the gravitational constant，and 
MP 一 S$.97 x 1024 kg is the mass of the earth. Note that (xl, x2) and (xz3,X4) 
denote the position and velocity, Tespectively, of the satellite on the plane 
having the earth at its origin. This state equation ls defined in the M-file 
“df _sat.m below. 


(a) Supplement the following program“nm6p05.m”which uses the three 
routines ode_RK4()，ode45()，and ode23() to find the paths of the 
satellite with the following initial positions/velocities for one day. 
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(b) 


(cb) 


function dx = df_sat(t,x) 
global G Me Re 
dx = zeros(Size(x)); 
sqrt(SsSum(x(1:2).^2)); 
if Fr <= Re，returnji end 55 when colliding against the earth Surface 
GMr3 = Gx*Me/r^3i 
dx(1) = xX(3); dx(2) = X(4); dx(3) = -GMr3*x(1); dx(4) = -GMr3x*x(2); 


%nm6p05.m to Solve a nonlinear d.e. on the orbit of a _ satellite 
ClLear，Clf 
global G Me Re 
G=6.67e-11; Me = 5.97e24; Re = 64e5 | 
千 "df sat' 
t0=0;)T= 24*60*60;j tf = TiN= 2000) 
R = 4.223e7; 
v20s = [3071 3500 2000] ; 
for iter = 1:]ength(v20s) 
xl10 = R; x20 = 0j v10 = 0;j v20 = v20s(iter) 
Xx0 = [x10 x20 v10 v20]; tol = 1e-6; 
[tR,xR] = ode_RK4(f,[to tf],x0;N); 


[t23Ss,Xx23S] = 0ode23Ss(f,[to tf],xo) ; 
plot(xR(:,1),xR(:,，2)，b' ，x45(:,1),，Xx45(:，2)，Kk.， 


plot(xR(:,1),xR(:,，2)，b' ，x45(:，1),，Xx45(:，2)，Kk. ， 
end 





GO (xio, xzo) = (4.223 x 107, 0)[m] and (xsao, x40) = (ulo, u20) 一 

(0，3071)[my/s]. 

(和 (xio, xz2o) = (4.223 x 10 ,0)[m] and (xcao, xzao) = (ulo, u20) 一 
(0，3500)[my/s]. 

(i 放 (xlo, xz2o) = (4.223 x 107, 0)[m] and (xso, xz4o) = (ulo, v20) = 
(0，2000)[my/s]. 

Run the program and check 夺 the plotting results are as depicted in 

Fig. P6.5. 


In Fig. P6.3，we see that the“ode23s()”Ssolution path differs from 
the others for case (ii) and the“ode45()”and“ode23s()”paths differ 
from the“ode_RK4()”path for case (ii). But，we do not know which 
one ls more accurate. In order to find which one is the closest to the 
true Solution,，apply the two routines“ode45()”and“ode23s()”with 
smaller relative error tolerance of tol = 1e-6to fnd the paths for the 
three cases. Which one do you think is the closest to the true solution 
among the paths obtained in (a)7 

The purpose of this problem is not to compare the Several MAILAB routines， 
but to warn the users of the danger of abusing them. With smaller number 
of steps (N) (i.e.,， larger step Size), the Toutine“ode_RK4()”will also deviate 
much from the true Solution. The MAILAB built-in routines have too many 
good features to be mentioned here. Note that setting the parameters Such as 
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Figure P6.5 The paths of a satellite with the same initial position and different initial velocities. 


the relative error tolerance (RelTol) is Sometimes very important for obtaining 
a ITeasonably accurate Solution. 


6.6 Shooting Method for BVP with Adjustable Position and Fixed Angle 


Suppose the boundary condition for a Second-order BVP is given as 
X (10) 二 X20， X(Ctr 一 XY17 (P6.6.1) 


Consider how to modify the MAILAB routines “bvp2_shoot()”and 
“bvp2_fdf()”So that they can accommodate this kind of problem. 


(a) As for “bvp2_shootp()”that you Should make, the variable quantity 
to adjust for Improving the approximate Solution is not the derivative 
X'(t), but the position x(io) and what should be made close to zero is 
still jx(o)) 三 XUr) 一 Xf. Modify the routine in such a way that x(10) 
js adjusted to make this quantity close to zero and make its declaration 
part have the initial derivative (dx0) instead of the initial position (x0) 
as the fourth input argument as follows. 


function [t,Xx] = bvp2_shootp(f,to,tf,dqxo,xf;,N,tol,kmax) 
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Noting that the initial derivative of the true Solution for Eq. (6.60.3) 1S 
Zero，apply this routine to solve the BVP by inserting the following 
Statement into the program“do_shoot .mm . 


[t,x1] = bvp2_shootp( 'df661' ,to,tf,0,xf,N,tol,kmax) ; 


and plot the resujlt to check if it conforms with that (Fig. 0.8) obtained 
by“bvp2_shoot ()”. 


(b) As for “pvp2_fdfp()”implementing the finite difference method, you 


have to approximate the boundary condition as 


站 局 放 aas 
X (10) 一 X20 一 二 一 X20， X_1 一 Xl 一 21X20， (P6.6.2) 
Substitute this into the finite difference equation corresponding to the 
initial time as 


7? 呈 一 X_ 

本 人 5 2 十 CGO0X0 Se 0 (P0.0.3) 
xl 一 2xo 十 X1 一 21x 
-一 一 一 一 十 al0x20 十 G00x0 一 10 

(aoo 太 一 2)xo 十 2x1 一 ja0 十 用 (2 一 paio0)x20 (60.0.4) 


and augment the matrix-vector equation with this equation. Also, make 
its declaration part have the initial derivative (dx0) instead of the initial 
Position (x0) as the Sixth input argument as follows: 


function [t,x]l = bvp2_fdfp(al,ao,utotf,dxo,xf,N) 


Noting that the initial derivative of the true solution for Eq. (6.60.10) 
jls 一 7, apply this routine to solve the BVP by inserting the following 
Statement into the program “do_fdf .m”. 


[t,x1] = bvp2_fdfp(al,a0o,utotf,-7,xf,N) ; 


and plot the result to check if it conforms with that obtained by using 
“bvp2_fdf()”and depicted in Fig. 0.9. 


6.7 BVP with Mixed-Boundary Conditions 工 


Suppose the boundary condition for a second-order BVP is given as 


X(10) 一 X10， CIX(Cr 十 c2X (7 一 C3 (P6.7.1) 


Consider how to modify the MAILAB routines “bvp2_shoot()”and 
“bvp2_fdf()”so that they can accommodate this kind of problem. 


(3) 


(b) 
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As for “bvp2_shoot()”that you Should modify, the variable quantity 
to adjust for improving the approximate solution ls Still the derivative 
X'(10),，but what should be made close to zero is 


Je (1o)) = clx(Gr) 十 coxz (tr) 一 c3 (P6.7.2) 


HI you don't know where to begin，modify the routine 
“bvp2_shoot()”in such a way that x“ (1o) is adjusted to make this 
quantity close to zero. Regarding the quantity (P6.7.2) as a function of 
X (io), you may feel as 让 you were going to solve a nonlinear equation 
Jo)) = 0. Here are a few hints for this job: 


Make the declaration part have the boundary coefficient vector cf 
= [cl c2 c3] instead of the fnal position (xf) as the fifth input 
argument as follows. 


function [t,x] = bvp2m_shoot(f,to,tf,xo;cf;,N,tol,kmax) 


Pick up the first two guesses of x“(10) arbitrarily. 
You may need to replace acouple of statements in “bvp2_shoot()”by 


= cfx*[x(end,:) -1]; 

= cfx*[x(end,:) ; -1]; 

Now that you have the routine“bvp2m_shoot()”of your own mak- 
ing，don't hesltate to try using the weapon to attack the following 
problem: 


1 
xD) 一 41x(Dx (CD 十 2x2(00 一 0 with x(0) = 下 2x() 一 3x(D 一 0 
(P6.7.3) 
For this job, you only have to modify one statement of the program 
“do_shoot”(Section 6.0.1) into 


[t,x] = bvp2m_shoot('df661' ,to,tf,x0,[2 -3 0] ,Ntol,kmax); 


Iyou run it to obtain the Same Solution as depicted in Fig. 0.8, you 
deserve to be proud of yourself having this book as well as MAILAB; 
otherwise, just keep trying until you Succeed. 

As for “bvp2_fdf()”that you Should modify, you have only to aug- 
ment the matrix-vector equation with one row corresponding to the 
approximate version of the boundary condition clx(tr) 十 c2XY' (1r) = c3， 
that is， 

XN 一 YN-1 


CI1XN 十 c2 一 0 一 C3; 一 C2XYN_1 十 (cl1 十 c?)XN 一 C37 (P0.7.4) 
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Needless to say, you Should increase the dimension of the matrix A 


to N and move the xN term on the right-hand side of the (CN 一 lj)th row 
back to the leftrhand side by incorporating the corresponding statement 
into the for loop. What you have to do with“pvp2m_fdf()”for this 
job ils as follows: 


Make the declaration part have the boundary coefficient vector cf = 
[cl1 c2 c3] instead of the final position (xf) as the Seventh input 
argument， 


function [t,x] = bvp2m_ fdf(al,a0,utotf,xo,cf,N) 


Replace Some statement by A = zeros(N;N). 

JIncrease the last index of the for loop to N-1. 

Replace the statements corresponding to the (CN 一 JUDth row 
equation by 


A(N,N-1:N) = [-cf(2) cf(1)*h + cf(2)]; b(N) = cf(3)*hj 


which implements Eq. (P6.7.4). 
Modify the last statement arranging the Solution as 


X= [xo trid(A,b) ] ; 


Now that you have the routine“bvp2m_fdf()”of your own making， 
don't hesitate to try it on the following problem: 


2 吧 
XZ (1) 十 7x 0) 一 (0) 一 0 with x(1) 一 S$,x(2) 十 xX 2) = 一 3 
(P6.7.3) 
FEor this job, you only have to modify one statement of the program 
“do_fdf.m” (Section 60.0.2) into 


[t,x] = bvp2m fdf(al,a0,uito,tf,xo,[1 1 3],N); 


You might need to increase the number of segments N to Improve the 
accuracy of the numerical Solution. 开 you run it to obtain the Same 
Solution as deplicted in Fig. 6.9,pbe happy with it. 


0.8 BVP with Mixed-Boundary Conditions 工 


Suppose the boundary condition for a Second-order BVP is given as 


colX(10) 十 co2x (fo) 一 c03 (P6.8.1a) 
CHFIX(Cr 十 cj2X (7 一 CH3 (P06.8.1b) 


Consider how to modify the MAITLAB routines“bvp2m_shoot()”and 
“bvp2m_fdf()”so that they can accommodate this kind of problems. 


(3) 


(\b) 


(C) 
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As for “bvp2mm_shoot()”that you Should make, the variable quantity 
to be adjusted for improving the approximate solution is x“ (io) or x(1o) 
depending on whether or not col 天 0，while the quantity to be made 
Close to zero is Still 


xz(o),x (Go)) 一 cfxGr) 十 cfP2x (tr) 一 c13 (P6.8.2) 


H 开 you don't have your own idea, modify the routine “pvp2m_shoot ()” 
in Such a way that x' (10) or x(o) is adjusted to make this quantity 
close to zero and xY(10) Or x'(10) is set by (P6.8.1a), making its decla- 
Tation as 


function [t,x] = bvp2mm_shoot(f,to,tf,cocf,N,tol,kmax) 


Where the boundary coefficient vectors c0 = [co1 c02 c03] and cf = 
[cf1 cf2 cf3] are Supposed to be given as the fourth and fifth input 
arguments, respectively. 

Now that you get the routine“bvp2mm_shoot()”of your own mak- 
ing, try it on the following problem: 


2 2 可 
和 十 1 (P06.8.3) 


with x(0) 十 6x' (0) = 0，x() +x GD =0 


X (D) E 





As for “pbvp2_fdf()”implementing the finite difference method，you 
only have to augment the matrix-vector equation with two ITowsSs 
corresponding to the approximate verslions of the boundary conditions 
colX(10) 十 co2X (10) 一 co3 and CrlIxz(r 十 cP2X' (Lp 二 Cj3， that is， 





光合 
CO1X0 十 c02 六 二 co3， (col7 一 co02)X0 十 co2X1 一 C03 刀 
(P6.8.4a) 
XN 一 光 N 一 1 
CHIXN 十 c12 人 3 一 CAXN-I 十 (Crl 有 十 cj2)XN 一 Cj3 肠 
(P6.8.4b) 


Now that you have the routine“bvp2mm_fdf()”ofyour own making， 
try it on the problem described by Eq. (P6.8.3). 
Overall, you will need to make the main programs like“nm6p08a.m” 
and “nm6p08b.m”that apply the routines “bvp2mm_shoot()” and 
“bvp2mm_fdf()”to get the numerical solutions of Eq.(P6.8.3) and 
plot them. Additionally，use the MAILAB routine“bvp4c()”to get 
another solution and plot it together for cross-check. 


6.9 Shooting Method and Finite Difference Method for Linear BVPs 


Apply the routines“bvp2_shoot()”,“bvp2_fdf()”， and“bvp4c()”to 
Solve the following BVPs. 
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snm6p08a.m: to Solve BVP2 with mixed boundary conditions 
(2t/t^2 + 1)*x' -2/(t^2+1)*Xx +t^2+1 
With Xx(0)+6x' (0) =0，x'(1) + x(1) = 0 
Shooting method 
inline('[x(2); 2x*(tx*x(2) - X(1))./(t.2 + 1)+(t.^2 + 1)] tx ); 
=0iltf= 1;N= 100; tol = 1e-8; kmax = 10; 
= [160];cf= [110];%coefficient vectors of boundary condition 
[tt,x_sh] = bvp2mm_shoot(f,to,tf,cocf,N,tol,kmax) ; 
plot(tt,x_ sh(:， 1)，b') 


s%nm6p08b.m: finite difference method 

al = inline('-2x*t./(t.^2+1)，，t')j a0= inline('2./(t.^2+1) ，t'); 
inline('t.^2+1' 七) 

t0=0Oiltf= 1;N= 500) 

cO= [160];cf= [110];%coefficient vectors of boundary condition 

[tt,x_fd] = bvp2mm_ fdf(a1,a0,utotf,co，cf,N) ; 

plot(tt,x fd r) 





y CD) = OO, yGCD), CD)) with y(Cxo) = yoyCp)=y1  (P6.9.0a) 


Plot the solutions and fll in Iable P6.9 with the mismatching errors (of the 
numerical solutions) that are defined as 


function err = err of sol de(df,t,x,varargin) 
5 evaluate the error of solutions of differential equation 
[Nt,Nx] = Size(x);j if Nt< Nx，XxX= X.'; [Nt;Nx] = Size(x);j end 
n1=2:Nt - 1; t=t(:); h2s= tn + 1)-t(n1-1); 
dxX (x(n1+ 1,:) - X(n1 - 1,:))./(h2s*ones(1,NX)); 
num= xn1+1,:)-2*x(n1,:) + xn1 - 1,:); den = (h2s/2).^2*ones(1,NX) ; 
d2X 
for 人 
for n1(1):n1(end) 
dfX feval(df,t(n),[x(n;m) dxn - 1,m)],varargin{:}); 
errm(n - 1;,m) = d2xGn - 1,m) - dfx(end); 
end 
end 
err=Ssum(errm.^2)/(Nt - 2); 


%nm6p09_1.m 

%y"-y'+yY = 3*e^2t-2Ssin(t) with y(0) 5 &y(2)=-10 
tc0=0ojltf=2;jyo=5jyf= -10;N 100; tol = 1e-6; kmax = 10; 
df = inline( [y(2); y(2) - yY(1)+3*exp(2*t)-2*Sin(t)] ty ) 
al -1;) a0=1;jU= inline('3*exp(2x*t) - 2*Sin(t) ， 七 '); 

Solinit = bvpinit(1inspace(to,tf,5),[-10 5]); s[1 9] 

fbc = :inline('[y0(1) - 5; yf(1) + 10]" ，y0'，yf ) 

5 Shooting method 

tic，[tt,y_sh] = bvp2_shoot(df,to,tf,yo,yf,N,tol,kmax); times(1) = toci 
% Finite difference method 

tic，[tt,y_fd] = bvp2_fdf(a1,a0,utotf,y0,yf,N); times(2) = toci 
5 MATLAB built-in function bvp4c 

Sol = bvp4c(df,fbc,solinit,bvpset('` RelTol ,1e-6) ); 

tic，y_bvp = deval(Sol,tt); times(3) = toc 

%5 Eror evaluation 

ysS=[y_sh(:,1) y_fd y_pvp(1,:) ]; plot(tt,yS) 
err=err_ of _ sol de(df,tt,ySs) 
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Table P6.9 Comparison of the BVP Solver Routines bvp2_shoot ()/bvp2_fdf() 


Mismatching Error 
BVP Routine (P6.9.0b) Times 
690 
2_sh 
695 -| 
kmax = 10 
ET 
kmax = 10 机 
2_sh 
69 ppasot 
kmax = 10 
| 
693) 
kmax = 10 本 
2_sh 
696 | east | | 
N = 100，tol =1e-6， 4.4 x 10-25 国光 
kmax =10 
EXT 间 国生 国 硬 加 因 





N 一 1 
1 
er 一 站 -21DO7GD 一 ADyCD 7Co) uc) (P6.9.0b) 
1 一 1 
with 
一 2v(x， 洛 5 过 到 
而 网 5 二 y(Cxi+I) JS U 让 JE 1) 
(P6.9.0c) 
XF 一 X0 
Xi 一 X0 十 77， 刀 一 SS (P6.9.0d) 


and can be computed by using the following routine “err_of_sol_de() 7 
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Overall]，which routine works the best for linear BVPSs among the three 


routines? 

(a]) 光 G) = yy) 一 yo 十 3e 和 一 2sinx with y(0) =S,y(02) = 一 10 
(P6.9.1) 

(pb) y (xz) = 一 4y(x) with y(0) 一, y() = 一 (P6.9.2) 

(co) 交 G) = 10-5y(D 十 10-( 手 一 500) with y(0) = 0, y(50) = 0 (P6.9.3) 

(d) y (0 = 一 2y() 十 Sinft with y(0) =0,y(G) =0 (P06.9.4) 


(ej (xz) 一 yGC) 二 yy) 二 er(l 一 2x) with y(0) = 1,y() = 3e (P6.9.5) 
4 0 1 dy(C) 

旧 一 

7 GT 








=0 with yd =Inly2)=Imn2  (P6.9.6) 


06.10 Shooting Method and Finite Difference Method for Nonlinear BVPs 
(a) Consider a nonlinear boundary value problem of solving 


dl2 人 
和 2 一 7)， 刀 = 400 (P6.10.1) 
此 


with the boundary condition 7(xo) = 70，7Gxr) 三 了 


to fnd the temperature distribution 7(xz) [KK] in arod 4 m long, where 
[xo0， Xf] 一 [0， 4|]. 

Apply the routines“bvp2_shoot()”，“bvp2_ fdf()” and “bpbvp4c()” 
to solve this differential equation for the two sets of boundary conditions 
{Z(0) = 3900，7 (4) = 300} and {1Z (0) = 530，7 (4) = 300} as listed in 
Table P6.10. Fill in the table with the mismatching errors defined by 
Eq. (P6.9.0b) for the three numerical solutions 


%nm6p10a 
ClLear，Cclf 
K = 1.9e-9; Ta = 400; Ta4 = Ta^4; 

= inline('[T(2); 1.9e-9x*(T(1).^4-256e8)]'，t'，T'); 

=0;j xf=4; TO =500;Tf =300;N=500; tol = 1e-5; kmax = 10; 
s% Shooting method 
[xx,T_sh] = bvp2_shoot(df,x0,xf,T0,Tf,N,tol,kmax); 
%5 Iterative finite difference method 
al=0;a0=0;) uU=TO+[I:N - 1]*(Tf - TO)/Ni 
for = 1:100 
Xx,T_fd] = 2 让 (8 全 0,Tf,N) ; 
uU = Kx*(T_ fd(2:N).^4 - Ta4); %RHS of (P6.10.1) 
二 > 1& 兴 G fd -Tfdo)Vnorm(T fd0) < to1l，1i，breaki end 
T fdo = T_ fd; 
end 
% MATLAB built-in function bvp4c 
Solinit = bvpinit(1Linspace(x0,xf,5),[Tf T0]); 
fbc = inline('[Ta(1)-500; Tb(1)-300]'，'Ta'，'Tb'); 
tic，Sol = bvp4c(df,fbc,solinit,bvpset('RelTol' ,1e-6) ); 
T_bvp = deval(So1l,Xxx); time_bvp = toci 
% The Set of three SolLutions 
于 证 下 坊间 大 下- 二 日 届 
s% Evaluates the errors and plot the graphs of the Solutions 
err = err_of_ sol de(df,xx;ys) 
Subplot(321)，p1lot(xx;,TS) 
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Table P6.10 Comparison of the BVP routines bvp2_shoot()/bvp2_fdf() 


Mismatching Time 
Boundary Condition Routine Error (P6.9.0b) (Seconds) 
To 
P6.10.1) with 克 = 400 
7(0) = 9300, 了 (4) = 300 
| 
P6.10.1) with = 400 
7(0) = 5330, 了 (4) = 300 
2 20xl | | 
es 
(P6.10.2) with 三 
0) 王 0, y(1) = 一 0 
人 汪汪 全 | ope0 
(P6.10.3) with 
Te | 
1) 三 4,y(2) 一 8 
To 
P6.10.4) with 
1) =1/3，y(4) = 20/13 
sa 本 生生 天 条 | 国 汪 和 
II 
P6.10.3) with 
一 T/2, y(2) = 一式 /4 
es 
(P6.10-6) with 本 
2) 一 2,y(8) = 1/4 
2 





{TOoo) 0: N] Cr = 加 十 太一 0 二 with N 二 500 


Note that the routine“bvp2_fdf()”should be applied in an iterative 
way to Solve a nonlinear BVP, because it has been fabricated to accom- 
modate only linear BVPSs. You may start with the following program 
“nm6p10a.m” Which routine works the best for the first case and the 
Second case, reSpectively? 
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(b) Apply the routines“bvp2_shoot ()”“bvp2_fdf()” and“bvp4c()”to 
Solve the following BVPSs. Fill in Table P6.10 with the mismatching 
errors defined by Eq. (P6.9.0b) for the three numerical solutions and 
plot the solution graphs if they are reasonable solutions. 


站 允 -eyz=0 with y(00) =0,y(G =0 (P6.10.2) 
站 > 

国 允 -77TO72=0 withyD=472)= (P6.10.3) 
》 


(地 ) 一 一 0 with >() 一 3 y(4) 一 了 (P6.10.4) 

(iv 交 一 zi) with y(0) = 王立 /2, y(C2) 一 区 /4 (P6.10.5) 
1 

(只 六 亲人 with y(2) 一 2, y(8) = 1/4 (P6.10.6) 
多 


Especially for the BVP (P6.10.6)，the routine“bvp2m_shoot ()” 
Or“bvp2mm_shoot()”developed in Problems 6.7 and 6.8 Should be 
used instead of “bvp2_shoot()”, Since it has a mixed-boundary con- 
dition 工 . 

(cb Originally, the Shooting method was developed for solving nonlinear BVPs， 
while the finite difference method is designed as a one-shot method for Solv- 
ing linear BVPS. But the finite difference method can also be applied in an 
iterative way to handle nonlinear BVPSs, producing more accurate Solutions in 
less computation time. 


06.11 Eigenvalue BVPs 


(a) A Homogeneous Second-Order BVP to an Eigenvalue Problem 
Consider an eligenvalue boundary value problem of solving 


yoJT+Tocy=0 (P6.11.1) 
with coly(xo) 十 cooy (xo) = 0，cjrlyCr) 十 cfP2yGCp) = 一 0 


to fnd y(x) for x e [xo,Xxr] with the (possible) angular frequency ww. 

In order to use the finite difference method，we divide the solu- 
tion interval [xo,xzr] into N subintervals to have the grid points Xi 三 
X0 十 友 三 Xo 十 if 一 X0)/N and then, replace the derivatives in the 
differential equation and the boundary conditions by their fnite differ- 
ence approximations (9.3.1) and (3.1.8) to write 


yi-1 一 2 和 十 yi+l 避 
二 


y 1 一 CC) 六 +y=0 with 入 = 大 (P6.11.2) 


妈 三 0 
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with 号 
1 一 》 -1 01 
co1y0 十 co 一 5 一 0 一 ) 1 = 三 人 一 2 十 y(P6.11.3a) 
02 
N+1 一 yN=-1 Cl 
CHFlyN 十 cp 一 0 一 yw+l 三 yN-1 一 2 


(P6.11.3b) 
Substituting the discretized boundary condition (P6.11.3) into (P6.11.2) 


yields 
(P6.11.39) 
7 1 一 2y0 十 太 三 一 人 70 一 一 一 一 


( 二 2 和 而 二 2 和 世 尖 的 (P6.11.4a] 
cC02 
yi-1 一 2 站 十 yl 三 一 人 yi 一 一 yl1 十 2 有 一 HI 一 和 7 
for 1=1:N_ 1 (P6.11.40D) 
(P6.11.3b) 
yw-1 一 27N 十 yw+l 三 一 人》N 一 一 一 人 
Cj 
三 六 .7 弟 (: 汪 2 纪 ) 二 (P6.11.4o) 
CF2 


which can be formulated in a compact form as 


2 一 21pcol/coz 一 2 0 0 0 y0 
一 1 2 一 | 0 0 y1 
0 一 1 2 一 ] 0 
0 0 一 1 2 一 1 yN_1 
0 0 0 一 2 2 十 21cjrl/cj2 yN 
y0 
y1 
一 入 
yN-1 
yN 
4y 一 入 y; [4 一 和 7y=0 (P6.11.93) 


For this equation to have a nontrivial solution y 天 0, 入 must be one of 
the eigenvalues of the matrix 4 and the corresponding eligenvectors are 
possible solutions. 


316 ORDINARY DIFFERENTIAL EQUATIONS 


function [x,Y,ws,eigvals] = bvp2_eig(x0,xf,co,cf,N) 
Use the finite difference method to Solve an eigenvalue BVP4: 
y "+W^2x*y = With co1y(x0) + coO2y '(x0) = 0，cfly(xf) + cf2y ' (xf) = 0 
Input: X0/Xf the initial/final boundaries 
CO/cf the initial/final boundary condition coefficients 
N -1 the number of internal grid points . 
Output : X the vector of grid points 
革 the matrix Composed of the eigenvector Solutions 
WS = angular frequencies corresponding to eigenvalues 
eligvals the eigenvalues 
if nargin < 5IN< 3，N= 3; end 
h= (xf - Xx0)/Ni h2 = hx*h;i X= XO+[0:N]*h; 
N1=N+1， 
if abs(c0(2)) < eps，N1 = N1 - 1; Al(1,1:2) = [2 -1]; 
else A(1,1:2) = [2*(1-co(1)/c0(2)*h) -2]; %5(P6.11.4a) 
end 
if abs(cf(2)) < eps，N1= N1 - 1; AN,N1 - 1:N1) = [-1 2]; 
else A(N1,N1 - 1:N1) = [-2 2*(1 + cf(1)/cf(2)*h)]; %5(P6.11.4c) 
end 
if N1 > 2 
form= 2:ceil(N1/2)，A(m'm- 1:m+1) = [-12 -1];， end ss%(P6.11.4b) 
end 
for m=ceil(N1/2) + 1:N1 - 1，A(m,:) = fliplr(A(N1 + 1 - m:)); end 
[V,LAMBDA] = eig(A); eigvals = diag(LAMBDA) ; 
[eigvals,I] = Sort(eigvals); 5 Sorting in the ascending order 
V=V(TIT); 
wS = sqrt(eigvals)yVhi 
if abs(c0(2)) < eps，Y zeros(1,N1); elseY= []; end 
Y= [YYV]; 
if abs(cf(2)) < eps,Y= [Y; zeros(1,N1)]; end 





Note the following things: 
。 The angular frequency corresponding to the eligenvalue 入 can be 


obtained as 
一 VAN/ao/ 户 (P6.11.0) 


。 The eligenvalues and the eigenvectors of a matrix 4 can be obtained 
by using the MAILAB command "“[V,D] = eig(A) . 

。 The above routine“bvp2_eig()”implements the above-mentioned 
Scheme to Solve the Second-order eigenvalue problem (P6.11.1). 


。JIn particular, a second-order eligenvalue BVP 
Yo)+woy=0 with >Co)=0,yGr)=0 (P6.11.7) 


cofresponds to (P6.11.1) with co = [col coz]=[1 0] and cr 三 
[cr cf] = [1 0] and has the following analytical solutions: 


大 元 


全 2 (P6.11.8) 
XF 一 X0 


y(O) 一 aSsinwx with w 一 
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(a) Eigenvector solutions for BVP2 (b) Eigenvector solutions for BVP4 


Figure P6.11 The eigenvector solutions of homogeneous second-orderandfourth-order BVPs. 


Now, use the routine“bvp2_eig()”with the number of grid points 
N =256 to solve the BVP2 (P6.11.7) with xo 三 0 and xr 三 2, fnd 
the lowest three angular frequencies (wi S) and plot the corresponding 
elgenvector Solutions as depicted in Fig. P6.11a. 
(b) A Homogeneous Fourth-Order BVP to an Eigenvalue Problem 
Consider an eligenvalue boundary value problem of solving 
dy 4 
. dy dy 
with y(xo) = 0， 了 Co) 一 0,y(Cr) = 0， 了 CC7) 一 0 


to find y(x) for x es [xo,xr] with the (possible) angular frequency w， 

In order to use the finite difference method，we divide the Solu- 
tion interval [xo,xzr] into N subintervals to have the grid points Xi 二 
X0 十 矿 三 X0 二 IGCC 一 ZN and then, replace the derivatives in the 
differential equation and the boundary conditions by their fnite differ- 
ence approximations to wiite 


yi-2 一 4y-1 十 6 一 4y+1 十 yi+2 
7 
yi 一 471+6y 一 471+ya = 和 = 庆 oO) (OP6.11.10) 


一 oo =0 


with 六 和 
加 = 0， 一 一 5 一 0 一 y 1 =- (611119) 
2 风尘 
yw 二 0， 0 一 yw 一 


(P6.11.11b) 
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Substitutingthe discretizedboundary condition (P6.11.11)into(P6.11.10) 
ylelds 


(P6.11.11a) 
7_1 一 4y0 十 6y1 一 47 十 妥 一 和 yl 


Sy1 一 472 十 妆 王 入 y1 


(P6.11.119) 
0 一 471 十 6y 一 4y 十 中 = 和 2 一 一 一 一 


一 471 十 067 一 473 十 驻 和 2 
关 一 4yX+H1I 十 0y+2 一 4y43 十 +H4 一 和 Xi+2 
for 7 一 1:N 一 $ (P6.11.12) 
(6.11.11b) 





yN-_4 一 4yw -3 十 6yNw -2 一 4yN_ 1 十 yN 一 入 yN_2 








yw-4 一 4yNw-3 十 OyN -2 一 4yN -1 一 入 yN-2 


(6.11.11b) 
yw-3 一 4yw-2 十 0yN-1 一 4yN 十 yN+l 一 和 7yN-1 一 一 一 一 





yw-3 一 4yw-2 十 3yN-1 一 入 yN-1 


which can be formulated in a compact form as 


5 -41 0 0 0 00 妨 了 
一 4 0 一 4 1 0 0 0 | y2 y2 

1 一 4 0 一 4 1 0 0 yY3 yY3 

0 5 0 一 入 

0 0 1 一 4 0 一 4 1 YN-3 | yN-3 

0 0 0 1 一 4 0 一 4 yYN-2 yN-2 

0 0 0 0 1 -4 5 | yN-1 | yN-1 

A4Ay 一 )y， [4- 和 my=0 (P6.11.13) 


For this equation to have a nontrivial Solution y 夭 0, 和 must be one 
of the eigenvalues of the matrix 4 and the corresponding eligenvectors 
are possible solutions. Note that the angular frequency corresponding 
to the eligenvalue 入 can be obtained as 


中 二 VAN/P (P6.11.14) 


(iD Compose a routine“bvp4_eig()”which implements the above- 


Imentioned Scheme to Solve the fourth-order eligenvalue Problem 
(P6.11.9). 


function [x,Y,ws,eigvals] = bvp4_eig(x0,xf,N) 


(C) 
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(ii) Use the routine“bvp4_eig()”with the number of grid points N 一 
250 to Solve the BVP4 (P6.11.9) with xo 三 0 and xj 三 2, fndthe 
lowest three angular frequencies (wi 7S) and plot the corresponding 
elgenvector Solutions as depicted in Fig. P6.11b. 

The Sturm-Liouville Equation 

Consider an eligenvalue boundary value problem of solving 


全 (GD 十 ro0)y 三 和 4C)7 with yxo) 一 0, 7) 三 0 
(P6.11.19) 
to find y(xz) for x es [xo,xr] with the (possible) angular frequency w， 
In order to use the finite difference method，we divide the Solu- 
tion interval [xo, xzr] into N subintervals to have the grid points 2 一 
X0 十 友 三 X0 十 IC 一 ZX0/N, and then we TIeplace the derivatives in 
the differential equation and the boundary conditions by their finite 
difference approximations (with the step Size 1 /2) to write 


Jr +MOyG +MN2 一 1 一 MO 一 NM on -oonyan 











2(07]2) 
万 人 一 六 万 1 一 .好 一 
(st+ 3 2 /人 (ax 2 中 +reom = enyen 
yi-1 十 已 太 十 ciy+I 一 和 人 forz 一 12,...， N 一 1 (P6.11.10) 
with 
Ji 一 有 /2 Ji 十 有 /2 rc) 
i 一 一 一 一， ci = 一 一， and 六 一 一 汪 一 Ci 
129 (5) 129g (2 ) 


(2Gi 
(P6.11.17) 


(iD Compose a routine “sturm()” which implements the above- 
mentioned Scheme to solve the Sturm-Liouville BVP (P6.11.19). 


function [x,Y,wseigvals] = Sturm(f,r,q,x0,Xxf;,N) 


(ii) Use the routine “sturm()”with the number of grid points N = 230 
to Solve the following BVP2: 


全 ((1 十 xz)y) 一 一 2 和 y with y(Co)=0,yG 站 =0 
(P6.11.18) 
Plot the eigenvector Solutions _ corresponding to the lowest three 
angular frequencies ( S). 
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Optimization involves fnding the minimum/maximum of an objective function 
J(x) Subject to Some constraint YE 9%. 开 there is no constraint for x to Sat- 
isfy 一 of，equivalently，9 jls the universe 一 then it is called an unconstrained 
optimization; otherwise, it ls a_ constrained optimization. In this chapter，we 
will cover several unconstrained optimization techniques Such as the golden 
Search method, the quadratic approximation method, the Nelder-Mead method， 
the steepest descent method, the Newton method, the Simulated-annealing (SA) 
method，and the genetic algorithm (GA). As for constrained optimization，we 
will only introduce the MAILAB built-in routines together with the routines for 
unconstrained optimization. Note that we don't have to distinguish maximization 
and minimization because maximizing 太 (x) ls equivalent to minimizing 一 三 xc) 
and So, without loss of generality, we deal only with the minimization problems. 


7.1 UNCONSTRAINED OPTIMIZATION [L-2, CHAPTER 7] 


7.1.1 Golden Search Method 


This method is applicable to an unconstrained minimization problem such that 
the solution interval [a, p] is known and the objective function (xc) ls unimodal 
within the interval; that is, the sign of its derivative 户 (x) changes at most once in 
[ca, p] So that 太 (x) decreases/increases monotonically for [a,x2?]/[x2,D]，where 
X2 ls the solution that we are looking for. The so-called golden search procedure ls 
Summarized below and is cast into the routine “opt_ gs()”. We made a MATLAB 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang,，Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 儿 Sons, Inc. 
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program“nm711.m"，whphich uses this routine to find the minimum point of the 
objective function 
Fo)= 人 2 一 218 一 1 (7.1.1) 


GOLDEN SEARCH PROCEDURE 


Step 1. Pick up the two points c=4 十 (1 一 randd = 十 77 inside the 
interval [ad, 中 , wherer = (VS 1D)/2 and 灵 一 六 一 4a. 

tep 2. 玉 the values of jx) at the two points are almost equal [ie.， 太 (ac) 祥 
J 太 (O)] and the width of the interval is sufficiently small (il.e., 灵 祥 0)， 
then Stop the iteration to exit the loop and declare x“ 一 Corx 一 d 
depending on whether j 太 (c) < JJ(d) or not. Otherwise,， go to Step 3. 


tep 3. 开 jc) < jd)， let the new upper bound of the interval 2 < d; oth- 
erwise，let the new lower bound of the interval a 和 c. Then，go to 
Step 1. 


function [xo,fo]l = opt_ gs(f,ab,r, ToLX,TolFun,k) 
h=b-ajirh= rrhic=b- rhyjd= ar+ rhi 
fc = feval(f,c); fd = feval(f,d); 
ifk<=01|(abs(h) < TolX&abs(fc - fd) < TolFun) 
if fc<= fd xo= cj fo = fc 
else Xxo= dj fo = fdi 
end 
if k == 0，fprintf('Just the best in given # of iterations' )，end 
el1Sse 
if fc< fd，[xo,fol = opt gs(f,a dr ToLX,TolFun,k - 1); 
else [xo,fol = opt gs(f,c,b,r,ToLIX,TolFun,k - 1); 
end 
end 


%nm711.m to perform the golden search method 

f711 = inline(` (Xx.*Xx-4).^2/18-1 ，Xx' ); 

a=0jb=3;jnrmz=(sdqrt(5)-1)/2; TolX = 1e-4; TolFun = 1e-4; MaxIter = 100; 
[xo,fo]l = opt_ gs(f711,ab,r,TolLX,TolLFun,MaxIter) 





Figure 7.1 shows how the routine“opt_gs()”Pproceeds toward the minimum 
point Step by step. 
Note the following points about the golden search procedure. 


。 At every iteration, the new interval width is 


D 一 c=p 一 (ca+( 一 7 一 oa)=rP or dd 一 aa=a+7rm 一 4 一 77 
(7.1.2) 
So that it becomes 7 times the old interval width ( 一 a 三 万). 
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Figure 7.1 Process for the golden search method. 


。 The golden ratio r ls fxed so that a point cl = D1 一 rji 一 六 一 27 in the 
new interval [c, p] conforms with gd 一 a 十 7 站 一 0 一 (1 一 站 2, that is， 


一 1 十 VI 二 4 一 ! 人 
一 1 一 产 +r 一 1=0， r= 二 一 盖 、、 (7.1.3) 


7.1.2 Quadratic Approximation Method 


The idea of this method ls to (a) approximate the objective function 太 x) by a 
quadratic function pz?(x) matching the previous three (estimated solution) points 
and (b) keep updating the three points by replacing one of them with the minimum 
point of p2(x). More Specifically, for the three points 


{(Co, jc 万) (2 万)} With xo < X1 < X2 


we fnd the interpolation polynomial pz(x) of degree 2 to fit them and replace 
one of them with the zero of the derivative 一 that is, the root of D?(x) = 0 [see 
Eq. (P3.1.2) in Problem 3.1]: 


万 一 z 芭 二 太一 2 上 + 户 G3 一 z) 


0 一 (7.1 
2{Jj0Gx1 一 22) 十 万 (xc 一 ZX0) 十 户 (xo 一 2)} 


X 一 X3 一 


In particular, 过 the previous estimated solution points are equidistant with an 
equal distance 灵 (i.e., X2 一 X1 一 Xl 一 X0 一 由 ),thenthis formula becomes 


人 2{Jj0G1 一 z2) 十 万 Go 一 X0) 十 户 (Cxo 一 2)} oa 





十 有 


二 (7.1.5) 


2( 一 四 十 2 万 一 思 ) 
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We keep updating the three points this way until |x2z 一 xzo| 久 0 and/or | jx2) 一 
Jrxo)| 之 0，when we stop the iteration and declare xa as the minimum point. 
The rule for updating the three points ls as follows. 


1. In case Xo < X3 < XY1，wWe take {Xo,X3, XI1} or {x3,Xl,X2} as the new set of 
three points depending on whether 太 (x3) < xi) or not， 


2. In case Xl < X3 < XY2，We take {X1,X3,X2} Or {Xo,Xl,X3} as the new set of 
three points depending on whether 太 (x3) < 太 (Cxcl) or not. 


This procedure, called the quadratic approximation method, ls cast into the 
MATITLAB routine“opt_quad()”, which has the nested (Tecursive call) structure. 
We made the MAILAB program“nm712.m”，which uses this routine to find the 
minimum point of the objective function (7.1.1) and also uses the MAILAB 
built-in routine“fminbnd()”to find it for cross-check. Figure 7.2 Shows how 
the routine“opt_quad()”proceeds toward the minimum point step by Step， 


(cf) The MAILAB built-in routine“fminbnd()”corresponds to“fmin()”in the MATI- 
LAB of verslion.9.X. 


function [xo,fo]l = opt quad(f,x0,TolLX,TolLlFun,MaxIter) 
%Search for the minimum of f(x) by quadratic approximation method 
If length(x0) > 2，Xx012 = Xx0(1:3); 
el1Se 
If Jength(x0) == 2，a = x0(1); b = Xx0(2); 
elSsSea= x0 - 10;b= Xxo+ 10; 
end 
xo012 = [a (at+ b)/2 b]; 
end 
fo012 = f(Xx012) ; 
[xo,fol = opt_ quad0(f,x012,f012,TolLX,TolLFun,MaxIter) ; 


function [xo,fo]l = opt quad0(f,x012,f012,TolLX,TolLFun,k) 
X0 Xx012(1); xl1 = Xx012(2); x2 = Xx012(3) 
fo0 = f012(1); f1 = f012(2); f2 = f012(3) 
nd= [fO- f2fl - fo f2 - fill]*[X1*xlf X2xx2 XO*xX0; xl x2 x0] '; 
X3 nd(1)/2/nd(2); f3 = feval(f,x3); %Eq.(7.1.4) 
工 f <=01|abs(x3 - xl1) < TolXx | abs(f3 - fl1) < TolFun 
Xxo = X3j fo = f3; 
if k== 0，fprintf(' Just the best in given # of iterations )，end 
elSe 
If x3 < X1 
if f3 < fl1，xol2 = [x0 x3 xl1]; fo012 = [fo f3 fl]; 
else Xx012 = [x3 xl x2]; fo012 = [f3 fl f2]; 
end 
else 
if f3 <= fl1，x012 = [xl1 x3 x2]; fo012 = [fl1 f3 f2]; 
else Xx012 = [x0 xl x3]; fo012 = [f0O fl f3]! 
end 
end 
[xo,fo]l = opt_ quad0(f,x012,f012,ToLX,TolLlFun,k - 1); 
end 
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Figure 7.2 ”Process of searching for the minimum by the quadratic approximation method. 


%nm712.m to perform the quadratic approximation method 
Clear，Clf 

f711 = inline(' (x.*x - 4).^21/8-1 ，X' ); 

a=0jb=3; Tolx= 1e-5; TolFun = 1e-8; MaxIter = 100; 


[xodq,fodq] = opt_quad(f711,[a b],TolXx,TolIFun,MaxIter) 
%minimum point and its function value 
[xob ,fob] = fminbnd(f711,a,b) %MATLAB built-in function 





7.1.3 Nelder-Mead Method [W-8] 


The Nelder-Mead method is applicable to the minimization of a multivariable 
objective function, for which neither the golden search method nor the quadratic 
approximation method can be applied. The algorithm ofthe Nelder-Mead method 
Summarized in the box below is cast into the MATLAB routine“Nelder0()”. 
Note that in the NMW-dimensional case (NM > 2), this algorithm Should be repeated 
for each subplane as implemented in the outer routine“opt_Nelder()”. 
We made the MAILAB program “nm713.m ”to minimize a two-variable objec- 
tive function 
Jo xza) 一 好 一 xx2 一 4 十 2 一 2 (7.1.0) 


whose minimum can be found in an analytical way 一 that is, by setting the partial 
derivatives of 太 (xl, xz) with respect to X1 and x2 to zero as 





0 
= 一 Ja,x2)=271 一 2 一 4=0 
OX1 


Xo 一 (XY10, xz2o) 一 (3, 2) 





0 
= 一 jx2) 一 2 一 2 一 1=0 
0X2 
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NELDER-MEAD ALGORITHIM 


tep 7. Let the initial three estimated Solution points be w,，D and c，where 
Jo) < JO) < 三) 

Step 2. If the three points or their function values are sufficiently close to each 
other, then declare a to be the minimum and terminate the procedure. 

tep 3. Otherwise, expecting that the minimum we are looking for may be at 


the opposite side of the worst point c over the line dBD (see Fig. 7.3)， 
take 


ee 一 0 十 2(11 一 C)， where 1 一 (4 十 D)/2 


and 让 太 e) < jD), take e as the new ci otherwise, take 
7 盖 (1 十 e)/2 王 211 一 C 
and ijr < _ jc, take r as the new c; 寺 帮 r 门 二 JJ， take 
SS 一 (C 十 1)/2 


and 计 js) < 太 c), take sy as the new c; otherwise,， give up the two points 
2Dc and take 1 and cl =(a 十 c)/2 as the new D and c, reflecting our 
expectation that the minimum would be around a. 
teDn 4. Go back to Step 1 





弓 mm=(a+bD)/2 





r=m+(m-O 


e=m+20m-O 





Figure 7.3 Notation used in the Nelder-Mead method. 
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function [xo,fo]l = Nelder0(f,abc,fabc,TolLX,ToLlFun,k) 
[fabc,I] = sort(fabc); a= abc(I(1),，:); b= abc(I(2),，:);jc= abc(I(3)，:); 
fa = fabc(1); fb= fabc(2); fc= fabc(3);j fba = fb - fai fcb = fc -fb 
if k<=0|abs(fba) + abs(fcb) < TolFun | abs(b- a) +abs(c - b) < TolLX 
xo = aij fo =Tfai 
if k== 0，fprintf('Just best in given # of iterations' )，end 
el1se 
m= (at+tb)/2;) e= 3rm - 2*cj fe=TfTeval(f,e); 
iffe< fb c= 6ej fc= fei 
e1Sse 
(m+e)/2; fr = feval(fr) 
iffr<f cc=rj fc= fri end 
if fr >= fb 
S= (C+m)/2; fs= feval(f,s); 
iffs<Tf cc=Sj fc=Tfsi 
elseb=mic= (at+c)/2; fb = feval(f,b); fc= feval(f，c); 
end 
end 
end 
[xo,fo]l = Neldero0(f,[aib;icl,[fa fb fc]l,TolXx,TolFun,k - 1)) 
end 


function [xo,fo]l = opt_Nelder(f,x0,ToLX,TolFun,MaxIter) 

N = length(x0); 

ifN== 1s%for 1-dimensional case 
[xo,fo]l = opt_quad(f,x0,TolLIX,TolFun); return 

end 

S = eye(N); 

for IIL = 1:Ns%repeat the procedure for each Subplane 
il1=i+1iaifail>nN il= 1; end 
abc = [x0;j x0 + SGi,:); x0+ SGil,:)]; %each directional Subplane 
fabc = [feval(f,abc(1,:)); feval(f,abc(2,:)); feval(f,abc(3,:))]; 
[x0,fo]l = Nelder0(f,abc,fabc,TolLX,TolFun,MaxIter) ; 
ifN<3，breaki end %sNo repetition needed for a 2-dimensional case 


%nm713.m: do_Nelder 

f713 = inline('x(1)*(xX(1)-4-X(2)) +X(2)*(Xx(2)-1) ，Xx' ); 

x0 = [00]，TolXx = 1e-4; TolFun = 1e-9; MaxIter = 100; 

[xon,fon] = opt_Nelder(f713,x0,To1LX,TolLFun,MaxIter) 

s%minimum point and its function Value 

[xos,fos] = fminsearch(f713,x0) %5use the MATLAB built-in function 





This program also applies the MAILAB built-in routine “fminsearch()”to min- 
imlze the Same objective function for Practice and confirmation. The minimization 
process ls illustrated in Fig. 7.4. 


(cf) The MAILAB built-in routine“fminsearch()”uses the Nelder-Mead algorithm 
to minimize a multivariable objective function. It corresponds to“fmins()”in the 
MAILAB of version.5.x. 
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Figure 7.4 Process forthe Nelder-Mead method (nm713.m-opt_Nelder()). 


7.1.4 Steepest Descent Method 


This method searches for the minimum of an NMW-dimensional objective function 
in the direction of a negative gradient 


(7.1.7) 





了 
一 gCG) = -Vo = 一 已 叶 人 | 汪 史 | 
0OX1 0OX2 OXN 

with the step-size ak (at iteration K) adjusted So that the function value 1s 
Iminimized along the direction by a (one-dimensional) line search technique 
like the quadratic approximation method. The algorithm of the steepest descent 
method is Summarized in the following box and cast into the MAILAB routine 
“opt_steep()”. 

We made the MAILAB program “nm714.m” to minimize the objective func- 
tion (7.1.0) by using the steepest descent method. The minimization Process is 
jllustrated in Fig. 7.3. 





STEEPEST DESCENT ALGORITHM 


step 0. With the iteration number 上 一 0, fnd the function value 刀 = /Co) 
for the initial point X0. 

tep 17. Increment the iteration number K by one, find the step-Ssize wk_1 along 
the direction of the negative gradient 一 5&i_1 by a (one-dimensional) line 
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Search like the quadratic approximation method. 
QI 三 ArgMinu jl 一 %gL/lgeiD) (7.1.8) 


tep 2. Move the approximate minimum by the Step-Size xl along the direc- 
tion of the negative gradient 一 & -1 to get the next point 


XK 一 XI 一 QT1gk_1/ge1| (7.1.9) 





tep 3. 开 X 多 XI and 三 Xp 之 太 X iD),then declare xx to be the minimum 
and terminate the procedure. Otherwise， go back to Step 1 


function [xo,fo]l = opt_steep(f,x0,TolX,TolFun,alpha0,MaxIter) 
minimize the ftn f by the Steepest descent method . 
input: f = ftn to be given as a string 于 ' 
Xx0 = the initial guess of the Solution 
output: x0o = the minimum point reached 
fo = 下 (Xx(0)) 
nargin < 6，MaxIter = 100; end %maximum # of iteration 
nargin < 5，alpha0 = 10; end s%initial step Size 
nargin < 4，TolFun = 1e-8; end slf(x)| < TolFun wanted 
nargin < 3，TolX = 1e-6;j end %|x(k)- X(k - 1)I<TolLlX wanted 
X = X0;j fx0 = feval(f,x0); fx = fx0; 
alpha = alpha0; kmax1 = 25; 
Warning = 0; %the # of vain wanderings to find the optimum Step Size 
for k = 1: MaxIter 
g=grad(f,x);g= g/norm(g); %5gradient as a row Vector 
alpha = alphax*2; %for trial move in negative gradient direction 
fxl1 = feval(f,x - alphax*2xg); 
for kl1 = 1:kmax1 %find the optimum step size(alpha) by Line search 
fx2 = fx1; fxl = feval(f,X-alphax*g) ; 
if fx0 > fx1+TOLIFun & fxl1 < fx2 - TOlLFun %fx0 > fx1 < fx2 
den = 4*fx1 - 2*fx0 - 2*fx2j num = den - fx0 + fx2; %Eq.(7.1.5) 
alpha = alphaxnum/den; 
Xx=X- alpha*rgj fx = feval(f,Xx); %Eq.(7.1.9) 
break; 
else alpha = alpha/2; 
end 
end 
if kl >= kmax1，warning = Warning + 1; s%failed to find optimum step Size 
else warning = 0; 
end 
if warning >= 2|(norm(x - x0) < TolX&abs(fx - fx0) < TolFun)，breakj end 
x0 = xj fxo = fx; 
end 
XO = 
if K 


xj fo = fx; 
== MaxIter，fprintf('Just best in %d iterations' ,MaxIter)，end 


s%nm714 

f713 = inline('x(1)*(X(1) - 4 - X(2)) + X(2)*(X(2)- 1) ，Xx' ); 

x0 = [00]，Tolx = 1e-4; TolFun = 1e-9; alpha0 = 1; MaxIter = 100; 
[xo,fo]l = opt_steep(f713,x0,TolLX,TolFun,alpha0,MaxIter) 
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7.1.5 Newton Method 


Like the steepest descent method, this method also uses the gradient to Search for 
the minimum point of an objective function. Such gradient-based optimization 
methods are Supposed to reach a point at which the gradient is (close to) zero. 
In this context,， the optimization of an objective function JJ ls equivalent to 
fnding a zero of its gradient g(X)，which in general ls a vector-valued function 
of a vector-valued independent variable X. Therefore, if we have the gradient 
function g(CX) of the objective function 太 X),， we can Solve the System of nonlinear 
equations g(GX) 一 0 to get the minimum of JFX) by using the Newton method 
explained in Section 4.4. 

The backgrounds of this method as well as the steepest descent method can 
be shown by taking the TIaylor series of say,，a two-variable objective function 


CT 2): 
0O 6 2 
jx 72) 三 太 xlk, xz2k) 十 攻 本 世 | 
CCIK,X2K) 


OX1 0X2 X2 一 X2 大 


二 3/ax 89? 1 


3/3xz2gxl 927/3z 





[ze 一 xz az 一 2 | 


已 | 一 


攻 三 | 

CIKEX2K) X2 一 2K 
1 

Jo 兰 FGo 二 YAFGO7E 一 十 宁 区 = xk]7V2 CoOlx [一 xu] 


1 
Fo 兰 FGo) 十 中 区 一 区 ] 十 5 区 志 xkt 及 区 一 x] (7.1.10) 


with the gradient vector 中 = VJG)|x and the Hessian matrix 及 王 V2z 太 (X)|x 
In the light of this equation， we can see that the value of the objective function at 
point Xk+l updated by the Steepest descent algorithm described by 
Eq. (7.1.9) 


(0.1.9) 
XkHl 三 XK 一 QkgE/gt 
is most likely Smaller than that at the old point X，with the third term in 
Eq. (7.1.10) neglected. 
GD 兰 Fo 十 野 EXl 一 和 = Go 一 ok8 ge/lgtl 
J GeD -Go 兰 一 akgt gjllgtl <0 僵 JGeD < Go 


9Slightly different from this strategy of the steepest descent algorithm, the Newton 
method tries to go Straight to the zero of the gradient of the approximate objective 
function (7.1.10) 


(7.1.11) 





下 十 有 友 区 一 xj 一 0， X 一 欢 一 万 1gx (7.1.12) 


by the updating rule 
xktH1 一 允 一 万 gx (7.1.13) 


with the gradient vector 中 Volxk and the Hessian matrix 末 = V2FGolw 
(Appendix C 〇 ). 
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This algorithm is essentially to find the zero of the gradient function g(CX) of the 
objective function and consequently, it can be Implemented by using any vector 
nonlinear equation Solver. What we have to do is just to define the gradient 
function gCG) and put the function name as an input argument of any routine 
like“newtons()”or“fsolve()”for solving a System of nonlinear equations 
(See Section 4.0). 

Now，we make a MAILAB program “nm715.m”，which actually Solves 
SG 一 0 for the gradient function 


7 
gG) = VjG) = 攻 沁 | =[2x 一 zi2-4 2 一 2x 一 1 0.114) 
of the objective function (7.1.0) 


Jo = Jo za) 一 好 一 xl 一 4 十 好 一 各 


Figure 7.3 illustrates the process of searching for the minimum point by the New- 
ton algorithm (7.1.13) as well as the steepest descent algorithm (7.1.9), where the 
Steepest descent algorithm proceeds in the negative gradient direction until the 
minimum point in the line is reached，while the Newton algorithm approaches 
the minimum point almost straightly and reaches it in a few iterations. 


>>nm715 
xo = [3.0000 2.0000]，ans = -7 


%nm715 to minimize an objective ftn f(x) by the Newton method 

Clear，Clf 

f713 inline('x(1).^2 - 4*Xx(1) - X(1).*Xx(2) + X(2 
『 


) .2 - X(2) ，X ) 
1 ，xX'); 
= 50 


g713 inline('[2*x(1) - X(2) - 4 2*X(2) - X(1) 


x0 = [00]，Tolx = 1e-4; TolFun = 1e-6; MaxIte 
[xo,go,xx] = newtons(g713,x0,TolIX,MaxIter) ; 
Xxo，f713(xo) %an extremum point reached and its function value 














0 








Figure 7.5 ”Process for the steepest descent method and Newton method (nm714.m” and 
“nm715 .m). 
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了 R 了 emark 7.1， Weak Point of Newton Method. 

The Newton method is usually more efficient than the steepest descent method 
ionly it works as lllustrated above, but it is not guaranteed to reach the minimum 
point. The decisive weak point of the Newton method is that it may approach one 
of the extrema having zero gradient,， which ls not necesSarily a (local) minimum， 
but possibly a maximum or a Saddle point (See Fig. 7.13). 


7.1.6 Conjugate Gradient Method 


Like the steepest descent method or Newton method, this method also uses the 
gradient to search for the minimum point of an objective function，but in a 
different way. It has two versions 一 the Polak-Ribiere (PR) method and the 
Eletcher-Reeves (FER) method 一 that are Slightly different only in the search 
direction vector. This algorithm，summarized in the following box, ls cast into 
the MAILAB routine“opt_conjg()”， which implements PR or FR depending 
on the last input argument KC = 1 or 2. The quasi-Newton algorithm used in 
the MAILAB built-in routine“fminunc()”is Similar to the conjugate gradi- 
ent method. 

This method borrows the framework of the steepest descent method and needs 
abit more effort for computing the Search direction Vector S(1). It takes at most N 
iterations to reach the minimum point in case the objective function ls quadratic 
with a positive-definite Hessian matrix 如 as 


1 
JI) 一 5 ax 十 bzx 十 c ， where x: an NV-dimensional vector (7.1.19) 





CONJUGATE GRADIENT ALGORITHM 


Step 0 With the iteration number 大 一 0, fnd the objective function value 
如 = Jo) for the initial point Xo， 
sitep 17. Initialize the inside loop index, the temporary solution and the search 
direction vector to 灵 一 0,X() 一 Xe andS() 一 一 史 一 一 5CX), Tespec- 
tively，where g(X) ls the gradient of the objective function 三 (X)， 
tep 2. For 由 一 0to N 一 1repeat the following things: 
Find the (optimal) step-size 


on 一 ArgMin。 太 X(O2) 十 QS(D)) (7.1.10) 
and update the temporary solution point to 
X(7 十 1) 一 X(O2) 十 onS(D) (7.1.17) 
and the search direction vector to 


SO 十 了 三 一 gr+1 十 SOD) (4.1.18) 
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with 7 了 
忆 一 Se 人 p 交 (7.1.19) 
2 8 8 针 


9tep 3. Update the approximate Solution point to Xk+l 一 XCN)，which is the 
last temporary one. 


tep 4.IX SX land 帮 Xpo) 之 太 Xi),then declare xx to be the minimum 
and terminate the procedure. Otherwise, Increment 上 by one and go back 
to Step 1 


function [xo,fo]l = opt_conjg(f,x0,TolLX,TolFun,alpha0,MaxIter,KC) 
s#%KC = 1: Polak-Ribiere Conjugate Gradient method 
s%KC = 2: Fletcher-Reeves Conjugate Gradient method 
if nargin < KC = 0; end 
if nargin < MaxIter = 100; end 
if nargin < alpha0 = 10; end 
if nargin < TolFun = 1e-8; end 
nargin < TolX = 1e-6; end 
= length(x0); nmax1 = 20; warning = 0; h = 1e-4; %dimension of variab1e 
= X0; fxX feval(f,x0);j fx0o = fxj; 
= 1: MaxIter 
ko = fx;j alpha = alpha0; 
;Xih)i S= -9 
:N 
alpha0 ; 
eval(f,Xx + alpha*2x*S); %trial move in Search direction 
= 1:nmax1 s%To find the optimum Step Size by line Search 
fx1; fx1 = feval(f,x+alpha*xSs); 
xl1 + TolFun & fxl1 < fx2 - TolFun %5fx0 > fx1 < fx2 
*fx1 - 2*fx0 - 2*fx2; num = den-fx0 + fx2; %Eq.(7.1.5) 
alphax*num/den; 
pha*s;j fx = feval(f,x); 





n1 == nmax1/2 
= -alpha0; Tfxl = feval(f,Xx + alpha*x2x*S)j; 





= alpha/12; 


end 
Xx0 = xj fxo = fx; 
ifn<N 
g1 = grad(f,xih); 
if KC <=1，S= - g1+(g1 - g)*g1'/1(g*g' + 1e-5)*Si %(7.1.19a) 
elS8e S8= -g91 + gl1*g1'/1(g*g'+ 1e-5)*Si %(7.1.19b) 
end 
g = g1; 
end 
if n1 >= nmax1，warning = warning+1; scan't find optimum step Size 
else warning = 0| 
end 
end 
if warning >= 2|(norm(Xx - Xk0)<TolX&abs(fx - fk0)< TolFun)，breaki; 
end 
xo= Xij fo = fxi 
if k == MaxIter，fprintf('Just best in sd iterations' ,MaxIter)，end 


%nm716 to minimize f(x) by the conjugate gradient method . 

f713 = inline('x(1).^2 - 4x*x(1) - xX(1).*x(2) + X(2).^2 - X(2)，Xx'); 
x0 =[0 0]，Tolx = 1e-4; TolFun = 1e-4; alpha0 = 10; MaxIter = 100; 
[xo,fo]l = opt_conjg(f713,x0,TolIX,TolFun,alpha0,MaxIter ,1) 

[xo,fo] opt_conjg(f713,x0,TolIX,TolFun,alpha0,MaxIter,2) 
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Based on the fact that minimizing this quadratic objective function ls equivalent 
to Solving the linear equation 


gsGX) =VJGx)= 有 x+b=0 (7.1.20) 
MATITLAB has several built-in routines Such as“cgs()” “pcg()” and “picg()”， 
which use the conjugate gradient method to Solve a set of linear equations， 
We make the MAILAB program“nm716.m” to minimize the objective func- 
tion (7.1.0) by the conjugate gradient method and the minimization process is 
jllustrated in Fig. 7.0. 


7.1.7 _ Simulated Annealing Method [W-7] 


All of the optimization methods discussed so far may be more or less efficient 
in fnding the minimum point if only they start from the initial point sufftciently 
close to it, But, the point they reach may be one of Several local minima and we 
often cannot be Sure that it is the global minimum. How about repeating the pro- 
cedure to search for all local minima starting from many different Initial guesses 
and taking the best one as the global minimum'y? This would be a computation- 
ally formidable task，Ssince there is no Systematic way to determine a suitable 
Sequence of initial guesses, each of which leads to its own (local) minimum So 
that all the local minima can be exhaustively found to compete with each other 
for the global minimum. 

An interesting alternative ls based on the analogy between annealing and min- 
imization. Annealing is the physical process of heating up a solid metal above its 
melting point and then cooling lt down so Slowly that the highly excited atoms 
can settle into a (global) minimum enersgy state, yielding a single crystal with 
a regular Structure. Fast cooling by rapid quenching may result in widespread 














Figure 7.6 ” Process forthe conjugate gradient method (nm716 .m). 
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irregularities and defects in the crystal Structure，analogous to being too hasty 
to fnd the global minimum. The simulated annealing process can be imple- 
mented using the Boltzmann probability distribution of an energy level 无 (>0) 
at temperature 7 described by 


DP( 居 ) 一 wexp( 一 尼 /K7) with the Boltzmann constant K and w 一 1KT 
(7.1.21) 
Note that at high temperature the propbability distribution curve is almost fat over 
a wide range of 丈 , implying that the System can be in a high energy state as 
edqually well as in a low energy state，while at low temperature the probability 
distribution curve gets highervlower for lowerhigher 天, Implying that the System 
will most probably be in a low energy state,but Still have a Slim chance to be 
in a high energy state So that it can escape from a local minimum energy state. 
The idea of Simulated annealing is Summarized in the box below and cast 
into the MAILAB routine“sim_an1()”. This routine has two parts that vary 
with the iteration number as the temperature falls down. One is the Size of Step 
Ax from the previous guess to the next guess，which ls made by generating a 
Irandom vector y having uniform distribution UV [一 1 十 1H] and the same dimension 
as the variable x and multiplying -LI(y) (in atermwise manner) by the difference 
vector (u 一 D) between the upper bound u and the lower bound 1 of the domain 
of X. The /一 1-law 


_ (GTAOP 一 1 
8710) = 一 signO) forpolsl (7.1.22) 


implemented in the routine“mu_inv()”has the parameter that is increased 
according to a rule 


见 一 10100W) with g > 0: the quenching factor (7.1.23) 


as the iteration number K increases, Teaching 从 一 10100 at the last iteration 大 一 
Kmnax. Note the following: 


。 The quenching factor g > 0 18 made Smalllarge for Slow/fast quenching. 


。The value of /-L-law function becomes Small for |y| < 1 as M increases 
(See Fig. 7.7a). 


The other is the probability of taking a step Ax that would result in change 
Aj >0 of the objective function value jj). Similarly to Eq. (7.1.21),， this is 
determined by 


大 7 Ar 
P(taking the Step AX) 一 exp (- (去 Te for A >0 (7.1.24) 
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1 T 1 T T 
Ax= gu (J(u-) 
0.8 上 上 ith 间 0.8 忆 (AX) 一 
几 = 10100 (kmam) exp (一 (Kmnax)? AH|F(x)l/e 衣 
0.6 王 0.6 1 
5 
0.4 上 全 0.4 / 
, 惟 、 as Kiteration numben 人 
= 0.01- 人 人 ~、 increases 和 区 
1 六 ED02522A 了 记 | 0 as Kiteration numben 
| 从 = 1050 increases 
0 0.5 yy 1 0 AHj|F(x)l/ef 
(a) The mu-law inverse function 971(J) (b) The exponential function for 


randomness control 
Figure 7.7 _ 川 ustrative functions used for controlling the randomness-temperature in SA. 


SIMULATED ANNEALING 


Step 0. Pick the initial guess Xo, the lower bound 1 the upper bound u, the 
maximum number of iterations Kmnax > 0, the quenching factor g > 0 (to 
be made smallMlarge for Slow/fast quenching), and the relative tolerance 
sj of function value fuctuation. 


Sep LE 三 到 三 
tenp 2. For 大 一 1 to Kmnax,， do 


{Generate an N x 1 uniform random vector of UV [-1, 十 H] and transform 
it by the inverse AL law (with 人 = 王 10100 人 ne ) to make Ax and then 
take X1 < X 十 Ax, confining the next guess inside the admissible reglon 
{XlL<xXx<ul as needed. 


fAF = foo)- oo <0， 
{setX < XI andif FX) < setxX < 一 Xand 放生 FI) 
Otherwise， 


{generate a uniform random number z of U[0,1] and setX < XI only in case 


z < ptaking the step Ag) “ 兰 ”exp( 一 (K/nao04AF/IFGoOl/sy) 


} 


tep 了. Regarding X? as close to the minimum point that we are looking for， 
we may set X2 as the initial value and apply any (local) optimization 
algorithm to search for the minimum point of 三 (x). 
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function [xo,fol = Sim_anl(f,x0,1,uUkmax,q; ToLIFun) 
%5 Simulated annealing method to minimize f(X) st. 1 <= X <= U 
1Length(x0) ; 
X= X0; fx = feval(f,x); 
xoO = Xijfo= fxi; 
if nargin < 7，TolFun = 1e-8; end 
if nargin < 6 q= 1; end %squenching factor 
if nargin < 5，kmax = 100; end %maximum iteration number 
for kK 0:kmax 
TI (k/kmax)^qj %inverse of temperature from 0 to 1 
站 U 10^(Tix*100); 5 Eq.(7.1.23) 
dX mu_inv(2*rand(size(x))- 1;，mu).*(uU - 工 ) 
X1 X + dxj %next guess 
xl1= (x1< 1).*1l +(L1 <= X1).*(x1 <= U).*xxl +(uU< x1).x*xui 
%confine it inside the admissible region bounded by 1 and Uu.， 
fx1 = feval(f,x1); df = fx1- fx; 
if df < 0lIrand < exp(-Tixdf/(abs(fx) + eps)/TolFun) Eq.(7.1.24) 
X= Xi fx= fxil; 
end 
if fx < fo， xo= xijfo= fxli end 


function Xx = mu_inv(y，mu) 5 Inverse of mu-law Eq.(7.1.22) 
(((1+mu).^abs(y)- 1)V/mu).*sign(y); 


%nm717 to minimize an objective function f(x) by various methods , 

Clear，Clf 

f= inline('x(1)^4 - 16*x(1)^2 - 5*x(1) + X(2)^4 - 16*x(2)^2 - 5*X(2)  ，X' ); 
[-5 -5])UuU= [55]; %lower/upperbound 

x0 = [0 0] 

[xo_nd,fo]l = opt_Nelder(f,Xx0) 

[xos,fos] = fminsearch(f,x0) %cross-check by MATLAB built-in routines 

[xou,fou]l = fminunc(f,x0) 

kmax = 500; q = 1; TolFun = 1e-9; 

[xo_sa,fo_ sal = Sim_anl(f,x0,1，uUkmax,q, ToLFun) 





which remains as big as el for |AJ/Fo|l =s Fr at the last iteration K 一 Kmax， 
meaning that the probability of taking a step hopefully to escape from a local 
minimum and fnd the global minimum at the risk of increasing the value of 
objective function by the amount Aj = |J Colsr is still that high. The shapes of 
the two functions related to the temperature are depicted in Fig. 7.7. 

We make the MAILAB program“nm717.m7，which uses the routine“sim_ 
an1l()”to minimize a function 


Fo) 一 妇 一 16x1 一 5Sxl 十 xz 一 16xy 一 Sx2 (7.1.25) 


and tries other Toutines Such as“opt_ Nelder()” “fminsearch()” and“fmi- 
nunc()”for cross-checking. The results of running the Program are Summa- 
Tized in Table 7.1，which shows that the routine“sim_anl1()”may give us the 
global minimum even when Some other routines fail to fnd it. But，even this 
routine based on the idea of Simulated annealing cannot always Succeed and its 
Success/failure depends partially on the initial guess and partially on luck，while 
the Success/failure of the other routines depends Solely on the initial guess. 
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Table 7.1 Results of Running Several Optimization Routines with Various Initial Values 


X0 opt_Nelder() fminsearch() fminunc() Sim_anl() 

[0, 0] [2.9035, 2.90359] [2.9035,， 2.90306] [2.9036, 2.9036] [2.8966, 2.9036] 
( 太 = 一 1360.00) ( 太 2 = 一 136.00) ( 太 = 一 136.00) ( 太 = 一 136.600) 

[--0.$, 一 1.0] [2.9035, 一 2.7468] [一 2.7468, 一 2.7468] [一 2.7468, 一 2.7468] [2.9029, 2.9028] 
(大 = 一 128.39) (大 = 一 100.12) (大 = 一 100.12) (大 = 一 136.00) 


7.1.8 Genetic Algorithm [W-7] 


Genetic algorithm (GA) 1 a directed random search technique that is mod- 
eled on the natural evolution/selection process toward the Survival of the fittest， 
The genetic operators deal with the individuals in a population over Several 
generations to improve their fitness gradually. Individuals standing for possi- 
ble solutions are often compared to _ chromosomes and represented by strings of 
binary numbers. Like the Simulated annealing method，GA is also expected to 
fnd the global minimum solution even in the case where the objective func- 
tion has Several extrema, including local maxima, saddle points as well as local 
minima， 

A so-called hybrid genetic algorithm [P-2] consists of initialization，evalu- 
ation，Ieproduction (Selection)，crossover,， and mnutation as depicted im Fig. 7.8 


Initialize 
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Evaluation 
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for all the chromosomes 
are almost equal? 





Termination 
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L 7y 























Figure 7.8 Flowchart for a genetic algorithm. 


N=8,N=2,Nbo=[88] 


01100110 
01001111 
11110110 
01110111 
10101101 
11011011 
11011000 
10011100 


pool P 


01100110 


10101011 


01101000 


11101111 
10110011 


11110110 


00000001 


00011110 


random pairing 


al 01111100 
b1 01010111 
c1 11000010 
d1 10010011 
el 10101101 
f1 11000010 
g1 101oH1101 
h1 10100001 


al 01111100 
b1' 01000010 
cf1' 11001101 
d1 10010011 
el 10101101 
ff 11010111 
g1 10100010 
h1 10100001 


01111110 a2 
10101011 b2 
10011100 c2 
11001111 d2 
10110011 e2 
11010010 f2 
101100H11 g2 
01001010 h2 
crossovermutuation 


01111110 a2 
10101010 b2 
100111H1 c2， 
11001111 d2 
10110011 e2 
11010011 人 2 
10110000 g2， 
01001010 h2 


decode 
一 -人 


encode 
二 


decode 
一 
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population X 
-1.0000 -1.0000 
-1.9020 -1.7059 

4.6471 -0.9216 
0.3333 4.3725 
1.7843 2.0196 
3.5882 4.6471 
3.4706 -4.9608 
1.1176 -3.8235 
|， reproduction 
-0.1209 -0.0466 
一 1.5527 1.7356 
2.6259 1.1550 
0.7713 3.1452 
1.7843 2.0196 
2.6360 3.2601 
1.7843 2.0196 
1.3160 -2.0846 
-0.1373 “= 0.0588 
-2.4118 1.6667 
3.0392 1.2353 
0.7647 3.1176 
1.7843 2.0196 
3.4314 3.2745 
1.3529 1.9020 
1.3137 “” -2.0980 


fx 
-10.00 
-40.95 
44.67 


18.84 
luat 
evaluate -522 


19.71 
85.84 
一 12.54 


0.28 
-36.40 
一 50.62 
一 44.58 
一 54.22 
一 54.22 
一 35.76 


0.31 
一 46.12 
一 52.96 
一 44.73 
一 54.22 
一 43.50 
一 35.88 


Figure 7.9 Reproduction/crossover mutation in one iteration of genetic algorithm. 


and is Summarized in the box below. The reproduction/crossover process is illus- 


trated in Fig 7.9. This algorithm ls cast into the routine“genetic() 


辩 


and we 


append the following statements to the MAILAB program“nm717.m”in order 
to apply the routine for minimizing the function defined by Eq. (7.1.23). Inter- 
ested readers are welcome to run the program with these statements appended 
and compare the result with those of using other routines. Note that like the 
Simulated annealing, the routine based on the idea of GA cannot always Suc- 
ceed and its success/failure depends partially on the initial guess and partially 


on luck. 
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Nb 
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30; %population Size 
[12 12]; %the numbers of bits for representing each variab1e 
0.5; Pm 0.01; %Probability of crossover/mutation 


eta = 1; kmax 100; %5Learning rate and the maximum # of iterations 


[xo_gen,fo_gen] = genetic(f,x0,1,uU;Np,Nb,Pc,Pm,eta,kmax ) 





HYBRID GENETIC ALGORITHIM 


tep 0. Pick the initial guess xo 一 [xol1 ...Xov](Ni: the dimension of the vari- 


able), the lower bound 1= [0 .../v], the upper bound u = [xz ...XN]， 
the population Size N, the vector Np = [Mol ...Now] consisting of the 
numbers of bits assigned for the representation of each variable , the 
Probability of crossover P, the probability of mnutation P,, the learn- 
ing rate 1(0 <171 和]1, to be made Smalllarge for slow/fast learning)， 
and the maximum number of iterations Kmnax > 0. Note that the dimen- 
Sions of Xo,，uU, and 1 are all the Same as NMW，which is the dimension 
of the variable X to be found and the population size NW can not be 
greater than 2 in order to avoid duplicated chromosomes and should 
be an even integer for constituting the mating pool in the crossover 
Stage. 


tep 17. Random Generation of Initial Population 


Set X? 一 X0, j 丰 = 太 G2) and construct in a random way the initial pop- 
ulation array XI1 that consists of N states (in the admissible region 
bounded by u and D including the initial state X0o, by setting 


XI1() = xo and XI =1+rand*u--D fork=2:NDp (07.1.26) 


Where rand is a Tandom vector of the Same dimension N as X0，U， 
and 1]. Then, encode each number of this population array into a binary 
String by 


Ps1+ 和 NM: 和 ”Ni 


一 binary representation of Xi1(2, 7) with Wow bits 
XI10, 71) 一 CO) 
2U(11) 一 1(17) 
forz =1:Nnand 姑 三 1:N (7.1.27) 


=C ”一 D 


So that the whole population array becomes a pool array, each Iow of 
which is a _ chromosome represented by a binary String of 汉 1 Noi bits， 
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tep 2. For E 一 ] to [nax, do the following: 
1.， Decode each number in the pool into a (decimal) number by 


XK(02 ,711) 一 decimal representation of 


7111 一 1 11I 
已 ( 1 十 光 Ni : 交 由) with NM，bits 


&U() 一 7(00) 
一 用 (On ER 十 7(O) 
forz =1:Nnand 姑 三 1:N (7.1.28) 


and evaluate the value j 太 (2) of function for every Iow XK(02,:) 一 X(1) 
Corresponding to each chromosome and find the minimum in 三 太 2p) 
corresponding to XK(2D，:) 一 X(2D). 

2. 了 开 加 in 三 Jo) < 太 thenset 太一 Jo) andX? 一 XpD). 

3. Convert the function values into the values of fitness by 


亡 OD) = Max'2 fo) -AD (7.1.29) 


Which is nonnegative Y7 三 1:Nnandislarge for agood chromosome. 


N 
4. 开 Max ii{ 放 (OOD)) 宕 0，then terminate this procedure，declaring X? as 
the best. 
Otherwise, in order to make more chromosomes around the best point 
X(12) in the next generation,， use the reproduction rule 


站 (7.1.30) 
万 (2p) 


to get a new population Xk+l with XI :) 一 XO) and encode it to 
ITeconstruct a new pool array PH by Eq. (7.1.27). 

5. Shuffe the row indices of the pool array for random mating of the chro- 
Imosomes. 

0.， With the crossover probability 已 ,exchange the tail part starting from 
Some random bit of the numbers in two randomly paired chromosomes 
(rows of PTD with each other s to get a new pool array 天 时 


7. With the mutation probability 已 ,, reverse a random bit of each number 
Iepresented by chromosomes (Tows of 已 1) to make a new pool array 


忆 1. 
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function [xo,fo] = genetic(f,x0,1,uU;Np,Nb,Pc,Pm,eta,kmax) 
s%5 Genetic Algorithm to minimize f(X) st. 1 <= X <= U 
N = length(x0) ; 
if nargin < 10，kmax = 100; end s%# of iterations(generations) 
if nargin 9leta > 1leta <= 0，eta = 1; end %learning rate(0 < eta < 1) 
if nargin 8，Pm = 0.01; end s%probability of mutation 
if nargin 站 0.5;j end s%probability of crossover 
if nargin 6，Nb 8*ones(1;,N); end s%# of genes(bits) for each variab1le 
if nargin < 5，Np = 10; end s%population Size(number of chromosomes) 
%Initialize the population pool 
NNb = Sum(Nb) ; 
xo= Xx0(:)')1=1(:)')UuU=uU(:) 
fo = feval(f,xo); 
其 丰 让 汪汪 
forn=2:Np，X(n,:)=1+rand(size(x0)).*(uU - 1); end %Eq.(7.1.26) 
P = gen_encode(X;,Nb,1,uU); %5Eq.(7.1.27) 
for k = 1:kmax 
X = gen_ decode(P,Nb,1,uU); %5Eq.(7.1.28) 
forn= 1:NP，fXx(n) = feval(f,X(n,:)); end 
[fxb,nb] = min(fX); %5Selection of the fittest 
if fxb < fo，fTo = fxbj xo = X(nb,:); end 
fX1 max(fxSs) - fX; s%make the nonnegative fitness vector by Eq.(7.1.29) 
fxm = fX1(nb) ; 
if fxm < eps，return; end %terminate if all the chromosomes are equal 
s%Reproduction of next generation 
于 0P 党- NB 
X(n,:) = Xn,:) + etaxr(fxm - fx1(n))/fxnx*(X(nb,:) - X(n,:)); %Eq.(7.1.30) 
end 
P = gen_encode(X,Nb,1,U) ; 
%Mating/Crossover 
Is = Shuffle([1:Np]); 
for 站 = 三 起 27NP 
if rand < PCc，P(is(n:n + 1)，:) = Crossover(P(is(n:n + 1)，,:),Nb); end 
end 
s%Mutation 
mutation(P,Nb,Pm) ; 


< 
< 
< 
< 


function P = gen_encode(X,Nb,1;,U) 
s% encode a population(X) of state into an array(P) of binary strings 
Np=Size(X,1); s%spopulation Size 
N = length(Nb); #%dimension of the variable(state) 
for nm 二 Nb 
b2 = 0; 
for mm 1:N 
b1 b2+1; b2 = b2 + Nb(m); 
Xnm =(2^Nb(m)- 1)*(X(n;m) - LI(m))/(u(m) - 1L(m)); s#%Eq.(7.1.27) 
P(n,b1:b2) = dec2bin(Xxnm,Nb(m)); sencoding to binary strings 
end 
end 


function X = gen_decode(P,Nb,1,U) 

s%% decode an array of binary strings(P) into a population(X) of state 
Np = Size(P,1); s%spopulation Size 

N = length(Nb); #%qdimension of the variable(state) 

for n 1:Np 


for mm 1:N 
b1 b2+1;b2= bl+Nblm - 1; s%5Eq.(7.1.28) 
X(n,m) = bin2dec(P(n,b1:b2))*(u(m) - LI(m))/7(2^Nb(m) - 1) + LI(m); 
end 
end 





CONSTRAINED OPTIMIZATION 343 


function chrms2 = Crossover (chrms2,Nb) 
5 Crossover between two chromosomes 
Nbb = length(Nb) ; 
b2 = 0; 
1:Nbb 
b2+1; bi= bl+mod(floor(rand*Nb(m))，Nb(m)); b2 = b2+ Nb(m); 
tmp = chrms2(1,bi:b2); 
chrms2(1,bi:b2) chrms2(2,bi:b2); 
chrms2(2,bi:b2) tmp 


function P = mutation(P,Nb,Pm) < mutation 
length(Nb ) ; 
= 1:Size(P,1) 


0 
1:Nbb 

if rand < Pm 
bl1=b2+1;bi= bl+mod(floor(randx*Nb(m)),，Nblm)); b2= b2+Nb(m); 
P(n,bi) = ~P(n,bi) 

end 

end 
end 


function is = Shuffle(is) 各 Shuff1e 
N = length(is) ; 
forn=N:-1:2 
in = Ceil(rand*(n - 1)); tmp = is(in); 
is(in) = is(n); is(n) = tmp; %5sSwap the n-th element with the in-th one 
end 





7.2 CONSTRAINED OPTIMIZATION [L-2, CHAPTER 10] 


In this section，only the concept of constrained optimization is introduced. The 
explanation for the usage of the corresponding MAILAB routines ls postponed 
until the next Section. 


7.2.1 Lagrange Multiplier Method 


A_ class of common optimization problems supject to equality constraints may 
be nicely handled by the Lagrange multiplier method. Consider an optimization 
Problem with M equajlity constraints. 


Min 三 (0) (7.2.1a) 
j1(X) 

三 由 2 (7.2.1b) 
Pr 


According to the Lagrange multiplier method, this problem can be converted 
to the following unconstrained optimization Problem: 


1M 
Min 1(X, 入) 一 三 X) 十 和 rhox) 一 三 (X) 十 >》， 入 六 用 (X) (7.2.2) 


1 一 | 
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The solution of this problem, if it exists, can be obtained by setting the derivatives 
of this new objective function 7(X, 入) with respect to X and 入 to zero: 


8 8 7 E 攻 
去 !G= 去 JO 二 人 二 hGo 一 YJCOT+ 》 和 mVhnCo = 0 (7.2.39) 


110 一 1 


0 
了 革 !C 入 ) = ho =0 (7.2.3b) 


Note that the solutions for this System of equations are the extrema of the objec- 
tive function. We may know it they are minima/maxima, from the positive/nega- 
tive-definiteness of the second derivative (Hessian matrix) of !(X, 入 ) with respect 
to X. Let us see the following examples. 


了 emark 7.2. Inequality Constraints with the Lagrange Multiplier Method. 

Even though the optimization Problem involves inequallty constraints like 
8Sj(G) < 0, we can convert them to equality constraints by introducing the Cnon- 
negative) Slack variables 7 as 


8jCO 十 六 =0 (7.2.4) 


Then，we can use the Lagrange multiplier method to handle it like an equality- 
constrained problem. 


卫 xample 7.1. Minimization by the Lagrange Multiplier Method. 
Consider the following minimization problem subject to a Single equality con- 
Straint: 


Min Fo = 好 十 好 (E7.1.1a) 
St， jhO=x+z 一 2=0 (E7.1.1b) 


We can Substitute the equajlity constraint xz 一 2 一 Xl into the objective func- 
tion (E7.1.1a) so that this problem becomes an unconstrained optimization Prob- 
lem as 

Min Fo) 一 xz 十 (2 一 2 一 2x1 一 4x1 十 4 (E7.1.2) 


which can be easily solved by setting the derivative of this new objective function 
With respect to X1 to zero. 


(E7.1.1b) 
2 人 


8 
5 由 二 生生 二 小 X1 一 1 区 2 一 X1=1 (7.1.3) 
1 


Alternatively，we can apply the Lagrange multiplier method as follows: 


0.2.2) 


Min 17(x, 入 ) 好 十 2 十 CI 十 xz 一 2) (7.1.4) 
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5 T 
隐 =X+x-2=0 








0 = 对 + 准 















< os 0 上 
h(x)=x+x-2=0 卢 
0 
5 5 
5 
-5-5 -5 0 5 
(al) A mesh-shaped graph (b) A contour-shaped graph 


Figure 7.10 The objective function with constraint for Example 7.1. 


8 
了 Ge (全 9 2x 二 一 0， 二 一 /2 (E7.1.5a) 
必 ] 
8 
-一 1G 全 2 一 0， 三 一/2 (E7.1.5b) 
0OX2 
8 
也 1 为 4 全”% 阅 十 站 一 2 一 0 (E7.1.5o) 


G7459 2 7.45aD) 


2 十 总 /2 二 2 三 2 7(E7 工 前 


(7.1.5a) (E7.1.5b) 
下 X2 ”一 


/2 三 :1 一 和 /2 一 1 (Fig. 7.10) (7.1.7) 

In this example, the substitution of (linear) equality constraints 1s more con- 
venient than the Lagrange multiplier method. However, it 1s not always the case， 
as illustrated by the next example. 


上 Example 7.2. Minimization by the Lagrange Multiplier Method. 
Consider the following minimization problem subject to a Single nonlinear 
equality constraint: 


Min 太 GX) 一 xl 十 X2 (上 7.2.1a) 
st， Po 一 好 十 巡 一 2=0 (E7.2.1b) 
Noting that it ls absurd to substitute the equality constraint (上 7.2.1b) into 


the objective function (E7.2.1a)，we apply the Lagrange multiplier method as 
below. 


Min 1G, 四 《全 2 阅 十 如 十 MG 十 z) (E7.2.3) 
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-2 -2 


Figure 7.11 The objective function with constraint for Example 7.2. 


区 人 (E7.2.4a) 

ee 为 “ 侍 ” 1 上 +2Xx 一 0， 加 一 一 1/2 (E7.2.4b) 

1G )《 宕 ”好 十 好 一 2=0 (E7.2.4c) 

好 十 闻 下 2 2 9 (1122 二 (1222 一 2， 和 一 土 1/2 (BE7.2.5) 
x 9 _1/= 干 |， 如 9 -1/2= 二 1 (E7.2.0) 


Now, in order to tell whether each of these ls a minimum or a maximum， 
we Should determine the positive/negative-definiteness of the Second derivative 
(Hessian matrix) of !(X, 入 ) with respect to X. 


821/8x | 


2 和 0 
921/8x29x1 321/82x 


| (E7.2.7) 


This matrix lis positive/negative-definite 让 the Sign of 入 1 positive/negative. 
Therefore, the solution (xl, X2) 一 (一 1 一 1) corresponding to 和 人 = 1/21isa(ocal) 
minimum that we want to get, while the solution (xl, xz2) 一 (1, 1) corresponding 
to 和 一 一 1/2is a (local maximum (See Fig. 7.11). 


D2 
五 一 /ex 入 ) 三 | 


7.2.2 Penalty Function Method 


This method is Practically very useful for dealing with the general constrained 
optimization problems involving equality/inequajlity constraints. It 18 really 
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attractive for optimization problems with fuzzy or loose constraints that are not 
SO Strict with zero tolerance. 
Consider the following problem. 


Min 三 (0) (7.2.9a) 


1 (X) 8S10X) 
S.t， h(x) 一 一 0， gS(GX) 一 <0 (7.2.5b) 
1jMy(X) 3SLG) 


The penalty function method conslsts of two stepS. The first step ls to construct 
a new objective function 


AM 二 
Min 1C0 = GO 二 》 wm 及 GO 十》 unen(CO) (7.2.6) 


1 一 1 11 一 1 


by including the constraint terms in Such a way that violating the constraints 
would be penalized through the large value of the constraint terms in the objective 
function，while satisfying the constraints would not affect the objective function. 
The second step ls to minimize the new objective function with no constraints 
by using the method that ls applicable to unconstrained optimization problems， 
but a non-gradient-based approach like the Nelder method. Why don'"t we use 
a gradientrbased optimization method? Because the inequality constraint terms 
Un (smn(GX)) attached to the objective function are often determined to be zero as 
long as X Stays inslde the (permissible) reglion satisfying the corresponding con- 
Straint (gm (X) < 0) and to increase very Steeply (like VC8gm(GX)) 一 exp(emgm (GD)) 
as X goes out of the region; consequently, the gradient of the new objective func- 
tion may not carry useful information about the direction along which the value 
of the objective function decreaseS. 

From an application point of view, it might be a good feature of this method 
that we can make the welghting coefficient (ww and ew) on each penallizing 
constraint term either large or Small depending on how strictly it Should be 
Satisfied. 

Let us see the following example. 


了 kxample 7.3. Minimization by the Penalty Function Method. 
Consider the following minimization problem Subject to Several nonlinear 
inequality constraints: 
Min Fo = {GCa 十 1.5)2 十 50o2 一 1.7)27{Gc 一 1.4) 7 十 0.6(xz2 一 0.5) 人 
(上 7.3.1a) 
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一 X1 0 
一 X2 0 
St ，gG) 王 | 3xl1 一 XIx2 十 4x2 一 7 | 三 |0 (7.3.1b) 
2X1 十 X2 一 3 0 
3X1 一 4x2 一 4x2 攻 


According to the penalty function method, we construct a new objective func- 
tion (7.2.60) as 


Min 1Co = {O 十 1.39)2 二 50o5 一 1.7)2{Gc 一 1.4) 十 0.60o2 一 0.5) 分 


注 
十 》 or 人 gw) (7.3.2a) 


11 一 1 
where 


0 if gw(X) 和 0 (constraint Satisfied) 
exp(emgm(X)) 诺 gwm(X) > 0 (constraint violated) ” 


V7m 一 1， Vi (Sm GD)) AS | 


ev =1VY 姑 一 1 .5 (E7.3.2b) 


%nm722 for EX.7.3 
s%5 to Solve a constrained optimization problem by penalty ftn method . 
Clear，CLf 
f ='f722p 
Xxo=[0.4 0.5] 
TolXx = 1e-4; TolFun = 1e-9j alpha0 = 1; 
[xo_Nelder,fo_Nelder] = opt_ Nelder(f,x0) %5Nelder method 
[fc_Nelder,fo_Nelder,co_Nelder] = f722p(xo_Nelder) %5its results 
[xo_s,fo s] = fminsearch(f,x0) %5MATLAB built-in fminsearch() 
[fc _s,fo_ s,co_sS] = f722p(Xxo_S) %5its results 

s#%5 including how the constraints are Satisfied or violated 


XOo_Ssteep = opt_steep(f,x0,TolXx,TolFun,alpha0) s%steepest descent method 
[fc_steep,fo_steep,co_steep] = f722p(xo_steep) 5%its results 
[xo_u,fo_ ul] = fminunc(f,x0); 55 MATLAB built-in fminunc() 
[fc_u,fo_u,co_uU] = f722p(xo_U) 5%5its results 


function [fc,f,c]l = f722p(X) 
f=((x(1)+ 1.5)^2 + 5*(X(2)- 1.7)^2)*((x(1)- 1.4)^2 + .6x*(X(2)-.5)^2) 1 
c=[-x(1); -X(2); 3*x(1) - X(1)*x(2) + 4x*Xx(2) - 7 

2*Xx(1)+ X(2) - 3; 3*x(1) - 4*X(2)^2 - 4*X(2)]; #%constraint Vector 
v=[11111];e= [11111]'; %weighting coefficient Vector 
fcC=TfT+vxr((c> 0).*xexp(e.*c)); %new objective function 
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>> nm722 


Xo_Nelder .2118 0.5765 xo_steep .2768 0.5989 
fo_Nelder “5322 %min Value fo_steep .2899 %not a minimum 
co_Nelder -1.2118 Co_steep .2768 
.5765 匡 .5989 
-1.7573 %high margin .5386 


.0000 s%no margin .1525 %Vviolating 
.0000 s%no margin .0001 


1.2118 0.5765 Warning: .. Gradient must be provided 
0.5322 %min Value 


Maximum # of function evaluations 
exceeded ; 

xo_U = 1.2843 0.6015 

fo _uU = 0.2696 s%not a minium 





Note that the shape of the penalty function as well as the values of the 
welghting coefficients ls set by the users to cope with their own problems. Then， 
we apply an unconstrained optimization technique like the Nelder-Mead method， 
which is not a gradient-based approach. Here, we make the program “nm722.m”， 
which applies not only the routine“opt_Nelder()”and the MAILAB built-in 
TIoutine “fminsearch()”for cross-check, but also the routine “opt_steep()”and 
the MAILAB built-in routine“fminunc()”in order to Show that the gradient- 
based methods do not work well. To our expectation, the running results listed 
above and depicted in Fig. 7.12 show that, for the objective function (上 7.3.2a) 
augmented with the penalized constraint terms，the gradient-based routines 
“opt_steep()”and“fminunc()”are not So effective as the non-gradient- 
based routines“opt_Nelder()”and “fminsearch()”in fnding the constrained 





1.5 






O:opt_ Nelder() 
+:{fminsearch() 
: opt_steep() 

:fminunc() 























Figure 7.12 The contours for the objective function (E7.3.1a) and the admissible region 
satisfying the inequality constraints. 
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minimum，which is on the intersection of the two boundary curves corresponding 
to the fourth and fifth constraints of (上 7.3.1b). 


7.3 MATLAB BUILT-IN ROUTINES FOR OPTIMIZATION 


Inthis section, weapply several MAILAB builtrin unconstrained optimlzation rou- 
tines including“fminsearch()” ”and“fminunc() ”tothe Same problem, expecting 
that their nuances will be clarified. Our intention is not to compare or evaluate the 
performances of these Sophisticated routines, but rather to give the readers Some 
feelings fortheir functional differences.Wealso introducetheroutine “linprog()” 
Implementing Linear Programming (LP) Scheme and “fmincon()”designed for 
attacking the (most challenging) constrained optimization problems. Interested 
readers are encouragedto run the tutorial routines “optdemo” or “tutdemo”, which 
demonstrate the usages and performances of the representative built-in optimiza- 
tion Toutines Such as“fminunc()”and “fmincon()”. 


7.3.1 Unconstrained Optimization 


In order to try applying the unconstrained optimization Toutines introduced 
in Section 7.1 and See how they work，we made the MAILAB program 
“nm731_1.m”"，which uses those routines for solving the problem 


Min Fo = Gil 一 0.53)2C 十 D2 十 (2 十 D20o2 一 1 (7.3.1) 


where the contours and the (local maximum/minimumy/saddle points of this 
objective function are depicted in Fig. 7.13. 
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Figure 7.13 The contours, minima, maxima, and saddle points ofthe objective function (7.3.1). 
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%nm731_1 
sto minimize an objective function f(x) by various methods , 
Clear，CLf 
s%5 An objective function and its gradient function 
inline('(x(1) - 0.5).^2.x*(X(1) + 1).^2 + (X(2)+1).^2.*(X(2) - 1).2'，Xx')) 
go0 = '[2*(x(1)- 0.5)*(x(1)+ 1)*(2*x(1)+ 0.5) 4*(X(2)^2 - 1).*x(2)] '; 
g = inline(g0，Xx' ); 
x0 = [0 .0.5] %initial guess 
pt_Nelder(f,x0) s%min point，its ftn value by opt_Nelder 


[xon,fon] = 0 
[xos,fos] = fminsearch(f,x0) %min point，its ftn Value by fminsearch( 


) 
[xost,fost] opt_steep(f,Xx0) %min point，its ftn Value by opt_steep() 
TOJX 1e-4; MaxIter = 100; 
XoOnt Newtons (g,X0,TolLX,MaxIter) ; 
Xxont,f(xont) sminimum point and its function value by Newtons () 
[xocg,focg]l = opt_conjg(f,x0) s%min point，its ftn value by opt_conjg() 
[xou,fou]l = fminunc(f,x0) %min point，its ftn value by fminunc() 





Noting that it depends mainly on the initial value Xo whether each routine 
Succeeds in finding a minimum point, we Summarize the results of running those 
Ioutines with various initial values in Table 7.2. It can be seen from this table 
that the gradient-based optimization routines like“opt_steep()” “Newtons ()”， 
“opt_conj ()”， and“fminunc()”sSometimes get to a saddle point or even a 
maximum point (Remark 7.1) and that the routines do not always approach the 
extremum that is closest to the initial point. It is interesting to note that even 
the non-gradient-based MAILAB built-in routine “fminsearch()”may get lost， 
while our routine“opt_Nelder()”works well for this case. We cannot，how- 
eveI，conclude that this routine is better than that one based on only one trial， 
because there may be Some problems for which the MAILAB built-in routine 
works well, but our routine does not. What we can state over this happening is 
that no human work ls free from defect. 

Now，we will see a MATILAB built-in routine “lsdqnonlin(f,x0;,1;,U， 
options,p1,..)” which presents a nonlinear leastrsquares (NLLS) solution to 


Table 7.2” Results of Running Several Unconstrained Optimization Routines with 
Various Initial Values 


X0 opt_Nelder fminsearch opt_steep Newtons opt_conjg fminunc 

[0, 0] [一 1 3] [0.5, ]] [0.$, 0] [一 0.23, 0] [0.$, 0] [0.$, 0] 
(minimum) (minimum) (Saddle) (maximum) (Saddle) (Saddle) 

[0, 0.5] [0.5, 1]] [0.02, 1]] [0.3, 1]  [--0.23, 一 1] [0.9, ] [0.3,]] 
(minimumy) (losb) (minimum) (Saddle) _ (minimum) (minimumy) 


[0.4, 0.5] [0.$, ] [0.5$, ]] [0.$, ] [0.5, 一 H [0.$, ] [0.$, ] 
(minimum) (minimum) (minimum) (minimum) (minimum) (minimumy) 

[一 0.3, 0.53] [0.3, H [一 1 ] [--1 1  [--0.23, -JJ [一 上 3 [一 1, ]] 
(minimum) (minimum) (minimum) (saddle) minimum) (minimumy) 

[一 0.8, 0.3] [一 1 3] [一 1 3] [一 1, 了 [一 1 一 也 [一 1 H] [一 1 ]] 
(minimum) (minimum) (minimum) (minimum) (mninimum) (minimumy) 
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the minimization problem 


N 
Min 》， (Co (7.3.2) 


1 一 1 


The routine needs at least the vector or matrix function fx) and the initial guess 
X0 as jlts first and second input arguments，where the components of fx) 一 
[ 廊 人 及 CGI] are squared, summed, and then minimized over X. In order to 
learn the usage and function of this routine，we made the MAILAB program 
“nm731_2.m”，which uses it to fnd a Second-degree polynomial approximating 
the function 


= oo = (7.3.3) 


1 十 8x2 


Eor verification, the result of using the NLLS routine “lsdqnonlin()”is compared 
with that obtained from directly applying the routine“polyfits()”introduced 
in Section 3.8.2. 


>> nm731_2 
ao_1sq = [-0.1631 -0.0000 0.4653]，ao_fit = [-0.1631 -0.0000 0.4653] 


s%nm731 2 try using 1sqnonlin() for a vector-valued objective ftn F(X) 
ClLear，Clf 

N=3; a0= zeros(1,N); %the initial guess of polynomial coefficient Vector 
ao_1sq = lsqnonlin('f731 2',a0) %parameter estimate by lsqnonlin() 

Xx = -2+[0:400]/50; 咎 X 


= 1./(1+8*XXx.xXX) ; 
ao_fit = polyfits(xx,fx;N - 1) %parameter estimate by polyfits() 


function F = f731 2(a) 
%error between the polynomial a(x) and f(x) = 1/(1+8x^2) 
xx = -2 +[0:200]/50; F = polyval(a,Xxx) - 1./(1+8*XXx.xXxXx); 





7.3.2 Constrained Optimization 


Generally, constrained optimization ls Very complicated and difficult to deal with. 
So we will not cover the topic in details here and instead, will just introduce the 
powerful MAILAB built-in routine“fmincon()” which makes us relieved from 
a big headache. 

This routine 1$s well-designed for attacking the optimization problems Subject 
to Some constraints: 


function [cceq]l = f722cC(x) 
C= [-x(1); -X(2); 3*x(1) - X(1)*Xx(2) + 4*X(2)- 7; 


2x*x(1)+ X(2)- 3; 3*x(1)- 4*x(2)^2 - 4*X(2)]; s%inequality constraints 
cedq = []; %equality constraints 
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(Usage of the MAILAB 6.x built-in function“fmincon()”) 
[xo,fo,,.] = fmincon( 'ftn',x0,A,b,Aedq,bedq,1,U，nlcon' ,options,p1,p2，,) 


。 11DUAFeg1e1pt (atleastfour input arguments 'ftn' ,X0,Aand b required) 


'ftn' : an objective function Fx) to be minimized, usually deftined in an 
M-fle, but can be defined as an inline function，which will 
remove the necesSslty of quotes(''). 

: an initial guess Xo of the solution 

: alnear inequality constraints 4X < b; to be given as [] inot 
applied. 

: alinear equality constraints 4。X 王 ber; to be given as [] 让 not 
applied. 

: lowerupper bound vectors such that 1<X<uitobesgivenas [] 
inot applied, set 1(i) = -inf/u(i) = inf if x(i) is not 
bounded below/above. 

: anonlinear constraint function defined in an M-fle, Supposed to 
return the two output arguments for a given Xi; the first one being 
the LHS (vector) of inequality constraints cC(CX) < 0 and the 
Second one being the LHS (vector) of equajlity constraints 


cer(CX) 一 0; to be given as [] 让 not applied. 
options: used for setting the display parameter, the tolerances for xo and 
JCxoj, and so on; to be given as [] inot applied. For details， 
type “help optimset” into the MAILAB command window、. 
p1,p2,,: the problem-dependent parameters to be passed to the objective 
function 太 CX) and the nonlinear constraint functions c(X)，cer (X). 


。 OU1IDU 4718U11e11S 


XO : the minimum point (Xo) reached in the permissible region 
Satisfying the constraints 
fo : the minimized function value 三 (xo) 


%nm732_ 1 to solve a constrained optimization problem by fmincon() 
Clear，ClLf 

ftn='((Xx(1) + 1.5)^2 + 5*(X(2) - 1.7)^2)*((Xx(1)-1.4)^2 + .6x(X(2)-.5)^2) 
f7220 = inline(ftn，X'); 

x0 = [00.5] s%initial guess 

A= [];B=[];Aeq= []; Beq= []; %no linear constraints 

1 = -infx*ones(Size(x0));)U= infx*ones(Size(X0)); 5 no Lower/upperbound 
options = optimset('LargeScale'，off'); %just [] is OK. 

[xo_con,fo_con] = fmincon(f7220,Xx0,A,B,Aeq,Beq,1,U，f722c' ,options) 
[co,ceqo] = f722c(xo_con) 5 to See how constraints are . 
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Min 太 (7.3.4) 
St 4X < b，4oX=bo，cQOO 三 0，cer(G 人 0) =0and1<XxX<u (7.3.9) 


A part of its usage can be Seen by typing “help fmincon” into the MAILAB 
command window as Summarized in the above box. We make the MAILAB 
program “nm732_1.m", which uses the routine “fmincon()”to Solve the problem 
presented in Example 7.3. Interested readers are welcomed to run it and observe 
the result to check if it agrees with that of Example 7.3. 

There are two more MAILAB built-in routines to be introduced in this Section. 
One is 


"fminimax( ' ftn',，x0,Ab,Aedqbedq,1,U，nlcon' ,options,p1,，..) "， 


which is focused on minimizing the maximum among several components of 
the vectormatrix-valued objective function fox = [A 罗 有 GO] subject to 
Some constraints as described below. Its usage is almost the Same as that of 
“fmincon()”. 


Min{Max{ 轧 (9 (7.3.0) 
St 4X < b，4。X= boy，cGX) < 0，ceorG) =0，and1<X<U (7.3.7) 
The other is the constrained linear least-squares (LL9) routine 
"1sqlin(C,d,Ab,Aedqbeq,1,uU,Xx0,options,p1,，..)"， 
whose job ls to solve the problem 


Min|ICx 一 dl (7.3.8) 


st， 4X<b，4oxX=bo and 1<X<Uu (7.3.9) 


In order to learn the usage and function of this routine，we make the MAILAB 
program“nm732_2.m，which uses both“fminimax()”and “1sqlin()”to fnd 
a Second-degree polynomial approximating the function (7.3.3) and compares 
the results with that of applying the routine“lsdqnonlin()”introduced in the 
previous section for verification. From the plotting result depicted in Fig. 7.14， 
note the following. 


。 We attached no constraints to the“fminimax()”routine, So it yielded the 
approximate polynomial curve minimizing the maximum deviation from 
(0). 

。 We attached no constraints to the constrained linear least-squares Toutine 
“1sqlin()” either，So it yielded the approximate polynomial curve 
minimizing the sum (integral) of squared deviation from 三 (x)，which is 
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Chebyshev “一 7 一 
we 














| 
-2 -1.5 一 1 -0.5 0 0.5 1 1.5 2 


Figure 7.14 Approximation of a curve by a second-degree polynomial function based on the 
minimax, least-squares, and Chebyshev methods. 


the Same as the (unconstrained) least Squares solution obtained by using the 
routine“1lsqnon1lin()”. 


。 Another MAILAB puilt-in routine“1lsqnonneg()”gives us a nonnegative 
LS (NLS) solution to the problem (7.3.8). 


s%5nm732_ 2: Uses fminimax() for a Vector-valued objective ftn 咎 (X) 
Clear，CLf 

f= inline(' 1./(1+8*Xx.xXx) ，X' ); 

f73221 = inline('abs(polyval(aXx) - fx)"，a' ，X'，fx'); 

f73222 = inline('polyval(a,x) - fx'，a' ，Xx'，fx'); 

N=2;5the degree of approximating poLlynomial 

a0 = zeros(1,N + 1); %initial guess of polynomial coefficients 

XX -2+[0:200]'/150; %intermediate points 

千 X feval(f,Xx); s%5 and their function values f(xx) 

aom= fminimnax(f73221,a0,[],[],[],[],[],[],[],[],xx,fx) %fminimax SO1 
forn= 1:N+l1，C(:n) = xx.^(N+1 -nn); end 

ao0_11 1Ssdqlin(C,fx) %51Linear LS to minimize (Ca - fx)^2 With no constraint 
ao_ln Jsqnonlin(f73222,a0,[],[],[],xx,fx) %nonlinear LS 

c2 = cheby(f,N,-2,2) %Chebyshev polynomial over [-2,2] 

plot(Xxx,fx，:' ，Xx,polyval(ao_m;Xxx)，m'，Xxxipolyval(ao_11,Xxx)，Pr) 
hold on，plot(xx,polyval(ao_ln,Xxx)，b'，Xxxpolyval(c2,Xx)，- 

axis([-2 2 -0.4 1.1]) 





7.3.3 Linear Programming (LP) 


The linear programming (LP) scheme Implemented by the MAILAB built-in 
Ioutine 


"[xo,fo]l = Linprog(f,A,b,Aeq,Beq,1,U,X0,options)” 


is designed to solve an LP problem, which is a constrained minimization problem 
as follows. 


Min Fo = 下 x (7.3.10a) 


Subject to 4xX<b， 4。x=b and 1<X<U (7.3.10b) 


eg， 
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733 to solve a Linear Programming problem. 
In fxXx=-3*Xx(1)-2*Xx(2) St. AXx <= b，Aeq = bedq and 1 <= X <= 
0= [00]; sinitial point 
[-3 -2]; %the Coefficient Vector of the objective function 
[3 4; 21];)b= [7; 3]; s%sthe inequality constraint Ax <= b 


[-3 2]; beq = 2; %the equality constraint Aeqx*x = beq 


[xo_lp,fo_lp] = Linprog(f,A,b,Aeqybeq,1,U) 

cons_Ssatisfied = [A; Aeq]*xo_lp-[b; beq] %how constraints are Satisfied 
f733o=inline('-3*x(1)-2*x(2) ，'Xx' ); 

[xo_con,fo_con] = fmincon(f7330,Xx0,A,b,Aeq,beq,1,U) 





It produces the solution (column) vector xo and the minimized value of the 
objective function jxo) as its first and second output arguments xo and fo， 
where the objective function and the constraints excluding the constant term are 
linear in terms of the Independent (decision) variables. It works for such linear 
optimization problems as (7.3.10) more efficiently than the general constrained 
optimization routine“fmincon ()”. 

The usage of the routine“linprog()”ils exempliftied by the MAILAB pro- 
gram “nm733.m”"，which uses the routine for Solving an LP problem described as 


Min Fo = 人 fx= [-3 一 2][D xi] = 一 3xl 一 2x2 (7.3.11a) 





S.t， 
2 一 「2 
4Ax 一 | 3 14 鸭 攻 7|=b and 
2 1|L2J<|3 
E 0 | X1 10 本 
[中 ss= [可 =[-。 eam 
全 X2 
2.5 了 
玉 


0.5 | -7 





Xe=0 | 
XI FF0 
节 
-0.5 0 0.5 1 1.5 2 2.5 





0 














Figure 7.15 The objective function, constraints, and solutions of an LP problem. 
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Table 7.3 The Names of MATLAB Built-In Minimization Routines in MATLAB 5.x/6.Xx 


Unconstrained Minimization Constrained Minimization 
Minimization Non-Gradient- Gradient- Linear Nonlinear 
Methods Bracketing Based Based Linear Nonlinear LS LS Minimax 
MAILAB 5.x ”fmin fmins fminu ]p constr leastsq conls ”Iminimax 


MATLAB 6.x fminbnd fminsearch fminunc linprog fmincon lsqnonlin lsqlin fminimax 


The program also applies the general constrained minimization routine“fmin- 
con()”to Solve the Same problem for crosS-check. Readers are welcome to run 
the program and see the results. 


>> nm733 
xo_lp = [0.3333 1.5000]，fo_lp = -4.0000 
Cons_Ssatisfied = -0.0000 %5 <= 0(inequality) 
-0.8333 5 <= 0(inequality) 
-0.0000 5 = 0(equalLity ) 
xo_con = [0.3333 1.5000]，fo_con = -4.0000 


In this result, the solutions obtained by using the two routines“linprog()”and 
“fmincon()”agree with each other, satisfying the inequality/equajlity constraints 
and it can be assured by Fig. 7.15. 

In Table 7.3, the names of MAITLAB built-in minimization routines in MAT- 
LAB version 9.x and 6.x are listed. 


PROBLEMS 


7.1 Modification of Golden Search Method 


In fact, the golden search method explained in Section 7.1 redquires only 
one function evaluation per iteration，since one point of a new interval 
coincides with a point of the previous interval so that only one trial point 
ls updated. In spite of this fact the MAILAB routine“opt_ gs()”imple- 
menting the method performs the function evaluations twice per iteration. 
An Improvement may be initiated by modifying the declaration type as 


[xo,fo]l = opt gs1(f,aeyferl,b,r,TolIXx,TolIFunyk) 


So that anyone could use the new routine as in the following program， 
where its input argument list contains another point (e) as well as the new 
end point (b) of the next interval, its function value (fe), and a parameter 
(F1) Specifying 让 the point ls the left one or the right one. Based on this 
ljdea, how do you revise the routine“opt_ gs()”to cut down the number 
of function evaluations? 
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s%nm7p01.m to perform the revised golden search method 
f701 = inline(' Xx.*x(X-2) ，'Xx' ); 
a=0jb=3jrz= (sdqrt(5)-1)/2; 
= 1e-4; TolFun = 1e-4; MaxIter=100; 
Prh = PrFxh; 
a + _rh; 


了 
fc = f701(c);j fd = f701(d) ; 
if fc< fd，[xo,fol = opt gs1(f701,ay cfc,1 -dir ToLX,ToLFun,MaxIter) 
] 一 


elSe [xo,fo 
end 


opt_gs1(f701,c,d,fdirbir，TolXx,TolLlFun,MaxIter) 





7.2 Nelder-Mead, Steepest Descent, Newton, SA, GA and fminunc(), fmin- 


search() 
Consider a two-variable objective function 


Ja = 一 12x1 一 4 十 好 一 16x 一 5x2 (P7.2.1) 
一 20cos(Xl 一 2.9) coS(X2 一 2.9) 


whose gradient vector function is 


4z 一 24xl 一 4 十 20sin(xl 一 2.3) cos(xzz 一 2.9) 

423 一 32x2? 一 9 十 20cos(xl 一 2.3) Sin(x2 一 2.9) 
(P7.2.2) 
You have the MAILAB functions f7p02(), g7p02() defining the objective 
function 太 X) and its gradient function g(CX). You also have a part of the 


MAILAB program which plots a mesh/contour-type graphs for JFX). Note 
that this gradient function has nine zeros as listed in Table P7.2.1. 


SG 一 Y/ GO 三 


Table P7.2.1 Extrema (Maxima/Minima) and Saddle Points of the Function (P7.2.1) 


Points Signs of 92 /az 天 Points Signs of 92 /az 











(D [0.6965 一 0.1423] 二 三 (6) [一 1.6926 一 0.1183] 

(2) [2.5463 一 0.1896] (7) [一 2.6573 一 2.8219] m 
(3) [2.5209 2.9027] 下; 主 (8) [一 0.3227 一 2.4257] 

(4) [一 0.3865 2.9049] (9) [2.5216 一 2.8946] ， mm 
(5) [一 2.6964 2.9031] 


(a) From the graphs (including Fig. P7.2) which you get by running the 
(unfinished) program，determine the characteristic of each of the nine 
points，that 1$s，whether it is a local maximum(M)minimum(mn)，the 
global minimum(G) or a saddle point(S) which is a minimum with 
Iespect to one variable and a maximum with respect to another variable. 
Support your judgment by telling the Signs of the second derivatives of 
J(GX) with respect to xl and X2. 
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4 3 2 1 0 1 2 3 4 





Figure P7.2 The contour, extrema and saddle points of the objective function (P7.2.1). 


s%nm7p02 to minimize an objective ftn f(x) by the Newton method 
f= 'f7p02';g= 'g7p02'; 
1=[-4-4];u= [44]; 
xf1 = LI(1):.25:uU(1); x2 = 1(2):.25:uU(2); [X1,X2] = meshgrid(x1,x2); 
form= 1:1ength(x1) 
forn= 1:1length(x2)，Fln;,m) = feval(f,[xl1(m) x2(n)]); end 
end 
figure(1)，clf，mesh(X1,X2;,F) 
figure(2)，clLf， 
contour(x1,x2;,F,[-125 -100 -75 -50 -40 -30 -25 -20 0 50]) 


function y = f7p02(X) 
yY=Xx(1)^4 - 12*x(1)^2 - 4*x(1) + X(2)^4 - 16*x(2)^2 - 
-20*coS(x(1) - 2.5)*cosS(X(2) - 2.9) 1 


function [df,d2f] = g7p02(x) sthe 1st/2nd derivatives 
df(1) = 4*x(1)^3 - 24*x(1) - 4 + 20*sin(x(1) - 2.5)*cos( 
df(2) = 4*x(2)^3 - 32*x(2)-5 + 20*cos(Xx(1) - 2.5)*sin(x( 
d2f(1) 12*x(1)^2 - 24 + 20*cos(xX(1) - )*cos(x(2) - 
d2f(2) 12*x(2)^2 - 32 + 20*cos(x(1) - )*cos(X(2) - 


区 
.5 





8/axzi = 12x1 一 24 十 20cosGl 一 2.5) cos(z 一 2.9) 


(P7.2.3) 
0/axy 一 12x1 一 32 十 20cosGrl 一 2.3) cos(z 一 2.9) 

(b) Apply the Nelder-Mead method, the steepest descent method, the New- 
ton method, the Simulated annealing (SA), genetic algorithm (GA), and 
the MAILAB built-in routines fminunc(), fminsearch() to minimize 
the objective function (P7.2.1) and fll in Table P7.2.2 with the number 
and character of the point reached by each method. 
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Table P7.2.2 Points Reached by Several Optimization Routines 
Initial Point Reached Point 


本 
ao [an | | 


国 国 汪 
医 世 克基 本 四 | 辣 是 三 呈 瑟 二 国有 大 汪 汪 区 瑟 引 渍 国 
IE 到 汪 | 贡 汪 用 本 本 天 二 | 丽 汪 汪汪 本 而 和 昨 了 到 
0 om | | | | 
CD em | | | 
医 瑟 2 天 汪 | 辣 二 三 本 天 二 本 本 友基 汪汪 天 本 省 本 二 
[| | | om | | 





(c) Overall the point reached by each minimization algorithm depends on 
the starting point 一 that is, the initial value of the Independent variable 
as well as the characteristic of the algorithm. Fill in the blanks in 
the following sentences. Most algorithms Succeed to find the global 
minimum if only they start from the initial point (,，),(，),(，),or(，). 
An algorithm most possibly goes to the closest local minimum (3) 革 
launched from (, ) or (,), andilt may go to the closest local minimum 
(7) 让 launched from (,) or (,). Ilaunched from (,), it may go to 
one of the two closest local minima (7) and (9) and 夺 launched from 
(，), lt most possibly goes to the closest local minimum (9). But, the 
global optimization techniques SA and GA seem to work fine almost 
Iegardless of the starting point, although not alwaysS. 


7.3 Minimization of an Objective Function Having Many Local Minimay/ 


Maxima 
Consider the problem of minimizing the following objective function 


Min Foo = sin(1/)/CC -0.2) 十 0.U (P7.3.1) 


which is depicted in Fig. P7.3. The graph Shows that this function has 

infinitely many local minima/maxima around x 一 0 and the global mini- 

mum about x 一 0.2. 

(a) Find the solution by using the MAILAB built-in routine“fminbnd ()”. 
JIs it plausible? 

(b) With nine different values of the initial guess xo 一 0.1, 0.2,... ,0.9, use 
the four MAITLAB routines“opt_Nelder()” “opt_steep()”“fmin- 
unc()”, and“fminsearch()”to solve the problem. Among those 36 
tryouts, how many times have you got the right solution? 


10 
5 

0 
-5 
-10 
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0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 


























Figure P7.3 Thegraphoffoo = sin(100/(X - 0.2)2 + 0.1) havingmanylocal minima/maxima. 


7.4 


(c) With the values of the parameters setto 1 一 0, 三 19 三 1s1 三 10-?， 
Kmnax 一 1000 and the initial guess xo 一 0.1, 0.2, ...,0.9,use the SA 
(Simulated annealing) routine“sim_an1l()”to solve the problem. You 
can test the performance of the routine and your luck by running the 
routine four times for the same problem and fnding the probability of 
getting the right solution， 

( 册 With the values of the parameters set to 1 三 0, 三 1 NM =30, Ni 三 
12, 已 =0.3, 己 , = 0.01,7 = 1 Kanax 一 1000 andthe initial guess xo 一 
0.1, 0.2, ...,0.9, use the GA (genetic algorithm) routine “genetic()” 
to Solve the problem. As in (c), you can run the routine four times for 
the same problem and find the probability of getting the right Solution 
in order to test the performance of the routine and your luck. 


Linear Programming Method 


Consider the problem of maximizing a linear objective function 
Max Fox) =fx=[3 2 -1 xx 2 下 (P7.4.1a) 


Subject to the constraints 


3 一 2 0 X1 一 一 2 
4X 一 | -3 -4 0 x2 | 过 |-7|=hb and 
-2 -1 0| | >| -3 
(P7.4.1b) 
0 1 10 
1 一 | 0 二 X 一 | X2 所 10 | = 
0 .3 10 


Jessica ls puzzled with this problem，which is not a minimization but a 
maximization. How do you Suggest her to Solve it? Make the program that 
uses the MATLAB built-in routines “linprog()”and “fmincon()”to solve 
this problem and run it to get the solutions. 
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7.S Constrained Optimization and Penalty Method 


Consider the problem of minimizing a nonlinear objective function 


Minfx = -3x1 一 2xz2 十 MGz 一 2z 十 2)” (P7.5.1a) 


(CM :alarge positive numbper) 


Subject to the constraints 


站 | 


IV 1A 
WD 
te 
已 
怠 
中 | 
天 一 
sa 和 ai 
| 
入 
芭 
中 | 
大 一 一 
人 
iD 一 
L_ 1 
1 和 A 
r 一 一 国 
一 一 
sa 
[| 
中 | 
吕 


(P7.5.1b) 


(a) With the two values of the weighting factor M = 20 and 10,000 in 


(b 


\ 必 


the objective function (P7.5.1a)，apply the MAILAB built-in routine 
“fmincon()”to find the solutions to the above constrained minimiza- 
tion problem. In order to do this job, you might have to make the vari- 
able parameter M passed to the objective function (defined in an Mr-file) 
either through“fmincon()”or directly by declaring the parameter as 
global both in the main program and in the M-file deftining (P7.3.1a). In 
case you are going to have the parameter passed through“fmincon() 7” 
to the objective function，you should have the parameter included in 
the input argument list of the objective function as 


function f=f7p05M(x,M) 


-3x*x(1)-2*Xx(2)+Mx(3x*x(1)-2xXx(2)+2) .^2; 





Additionally, you Should give empty matrices ([]) as the ninth input 
argument (for a nonlinear inequajlity/equality constraint function "nonl- 
con ) as well as the 10th one (for "options”`) and the value of AM as 
the 11lth one of the routine“fmincon()”. 


xo = fmincon('f7p05M' ,x0,A,b,[],[],1,u,[],[],NM) 


For reference，type “help fmincon”into the MAILAB command 
window. 
Noting that the third (Squared) term of the objective function (P7.3.1al) 
has its minimum value of zero for 3x1 一 2x? 十 2 一 0and,thus, it actu- 
ally represents the penalty (Section 7.2.2) imposed for not Satisfying the 
equality constraint 

3x1 一 2x? 十 2 一 0 (P7.3.2) 


tell which of the solutions obtained in (a) ls more likely to satisfy this 
constraint and Support your answer by comparing the values of the 
left-hand side of this equality for the two Solutions. 
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(c) Removing the third term from the objective function and splitting the 
equajlity constraint into two reversed inequality constraints，we can 
modify the problem as follows: 


Min Foo = -3xl 一 22 (P7.5.3a) 


Subject to the constraints 


3 4 7 

一 2 ”一 1 Xl | 二 一 3 
3 _2 图 二 _2 and (P7.3.3b) 
3 一 2 过 一 2 


3 0 < 二 1 中 10 < 是 
“5 可 二 全 X2 | 一 | 10 和 
Noting that this fits the linear programming，apply the routine“1lin- 


prog()”to solve this problem. 
(d 


Je 


Treating the equality constraint separately from the inequality con- 
Straints，we can modify the problem as follows: 


Minoo = -3xl 一 22 (P7.5.4a) 


Subject to the constraints 


3 
3 4|| 和 0 
-2 -1|L22 
(P7.5.4D) 


Apply the two routines“1linprog()”and“fmincon()”to solve this 
problem and see if the solutions agree with the solution obtained in (c). 

(cf Note that in comparison with the routine “fmincon()”, which can solve a gen- 
eral nonlinear optimization problem, the routine“linprog()”is made Solely 
for dealing with a class of optimization problems having a linear objective 
function with linear constraints. 


7.6 Nonnegative Constrained LS and Constrained Optimization 
Consider the problem of minimizing a nonlinear objective function 


Minx ICx-- dl = [Cx-d7cx-d] (P7.6.1a) 


Subject to the constraints 


_ | 如 0|1_ 
x 一 他 > 扑 一 1 (P7.6.1b) 


where 


1 2 
C=|3 4|， du=|10.8 (P7.6.1o) 
5 1 
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(a) Noting that this problem has no other constraints than the lower bound， 
apply the constrained linear least-squares Ioutine“1lsqlin()”to fnd 
the Solution. 


(b) Noting that the lower bounds for all the variables are zeros, apply the 
MATITLAB built-in routine“lsqnonneg()”to find the solution. 


(c) Applythe general-purpose constrained optimization routine “fmincon()” 
to find the Solution. 


7.7 Constrained Optimization Problems 


Solve the following constrained optimization problems by using the MATI- 
LAB built-in routine“fmincon()”. 


(a) Minx 一 5Sx7 十 6xl 十 2 一 2X2 十 X3 (P7.7.1a) 


Subject to the constraints 


xz 十 X 一 23<0 X1 0 
x+Tx+Txz>6 and X=|2|>|0|=1 07.7.lb) 
X3 芝 9 X3 0 


Try the routine “fmincon()” with the initial guesses_ listed im 
Table P7.7. 


Table P7.7 ”The Results of Applying “fmincon()”with Different Initial Guess 


Initial Guess 
三 (CX) Remark (warning 2) 


No feasible solution (w) 


| 

| 

一 

| 

| me 
一 es 
II 
ET 
| 
me 
| 
| 
一 | 
| 


Not a minimum, but the max 


Weird (warning) 
[10.25 0] 


1 
Cs Ta 
ET 
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(bl1) Maxx X1X2X3 (P7.7.2a) 


Subject to the constraints 


式 革 0 
X1X2 十 XY2X3 十 X3xl1 一 3 and X= | | 二 |0 (P7.7.2b) 
X3 0 
Try the routine“fmincon()”with the initial guesses listed in 
Table P7.7. 
(b2) Minx X1X2X3 (P7.7.3al) 


Subject to the constraints (P7.7.2b). 
Try the routine“fmincon()”with the initial guesses listed in 
Table P7.7. 


(c1) Maxx X1X2 十 X2X3 十 X3X1 (P7.7.4a) 


Subject to the constraints 


1 0 
Xl1 十 xz 十 3 一 3 and X= | |> 过 |0 (P7.7.4b) 
3 0 


Try the routine“fmincon()”with the initial guesses listed in 
Table P7.7. 


(Cc2) ”Minx XZ1X2 十 X2X3 十 X3X1 (P7.7.3a) 


Subject to the constraints (P7.7.4b). 
Try the routine“fmincon()” with the initial guesses listed in 


Table P7.7. 
10000 
() Minx 一 一 一 (P7.7.6a) 
1X72 
Subject to the constraints 
| (P7.7.6b) 
1 2 疙 0 


Try the routine“fmincon()” with the initial guesses listed in 
Table P7.7. 
(e) Does the routine work well with all the initial guesses? If not, does 
matter whether the starting point ls inside the admissible region? 

(cf Note that, in order to solve the maximization problem by“fmincon()”，we 
have to reverse the sign of the objective function. Note also that the objective 
functions (P7.7.3a) and (P7.7.Sa) have infinitely many minima having the value 
JI) 一 0inthe admissible region Satisfying the constraints， 
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(cf One might be disappointed with the reliability of the MAILAB optimization 
IToutines to see that they may fail to find the optimal solution depending on the 
inlitial guess. But, how can a human work be perfect in this world? It implies 
the difticulty of nonlinear constrained optimization problems and can never 
impair the celebrity and reliabijlity of MAILAB. Actually, it demonstrates the 
importance of studying Some numerical stuff in addition to just getting used 
to the various MAILAB commands and routines. 

Here is a tip for the usage of “fmincon()”: it might be better to use with 
an initial guess that is not at the origin, but in the admissible region Satisfying 
the constraints,， even though it does not guarantee the Success of the routine. 
It might also be helpful to apply the routine with Several values of the initial 
guess and then choose the best result. 


7.8 Constrained Optimization and Penalty Method 


Consider again the constrained minimization problem having the objective 
function (E7.3.1a) and the constraints (上 7.3.1b). 


Min 太 oo = {Gai 十 1.5)2 十 02 一 1.7)2{Gxl 一 1.4)2 十 0.60c 一 0.5)] (P7.8.1a) 


二 0 
一 X2 0 
st， gG) 王 | 3xl 一 xlxzz 十 4x2 一 7 | 和 | 0 (P7.8.1b) 
2X1 十 X2 一 3 0 
3X1 一 423 一 4x2 0 


In Example 7.3，we made the MAILAB program“nm722.m” to Solve the 
problem and deftined the objective function (E7.3.2a) having the penalized 
constraint terms in the fle named “f722p.m7”. 


Min /Co = {Gl 十 1.3)7 十 50 一 1.7)2{Co 一 1.4)2 十 0.60c2 一 0.5) 人 
+ 和 ww 人 rn) (P7.8.23) 


Where 


0 it gmw(X) < 0 (constraint Satisfied) 
exp(emgm(X)) 1 寺 gm(X) > 0 (constraint viloated) 


Von (sm CD)) 一 | 
和 本 让 (P7.8.2b) 


(a) Whatisthe weighting coefficientvectorvintheflenamed “f722p.m27Do 
the _ points reached by the routines “fminsearch()”/ “opt _ 
steep() /fminunc()”Ssatisfy all the constraints so that they are in the 
admissible regiony? Ifnot, specify the constraint(s) violated by the points. 


(b) Suppose the fourth constraint was violated by the point in (a). Then， 
how would you modify the weighting coefficient vector v So that the 
Violated constraint can be paid more respect? Choose one of the fol- 
lowing two weighting coefficient vectors: 
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(iD v [1111/31] 

(ii) v [11131] 

and modify the fle“f722p.m”with this _ coefficient vector. Then, run 
the program“nm722.m, fll in the 22 blanks of Iable P7.8 with the 
results and see if the fourth constraint is still violated by the points 
Ieached by the optimization routines? 

(c) Instead of the penalty method, apply the intrinsically constrained opti- 
mlzation Toutine“fmincon()”with the initial guesses Xo 一 [0.4 0.5] 
and [0.2 4] to Solve the problem described by Eq. (E7.3.1) or (P7.8.1) 
and fll in Table P7.8 with the results concerning the reached point and 
the corresponding values of the objective/constraint functions，. 

(dJ) Based on the results listed in Table P7.8, circle the right word in each 
of the parentheses in the following sentences: 

。 For penalty methods, the non-gradient-based minimization routines like 
“Nelder() 2”/ 六 fminsearch()”may work (better, worse) than the gradient- 
based minimization routines like“opt_steep() ”fminunc ()”. 

。 If some constraint is violated，you had better (increase，decrease) the 
Corresponding weight coefficient. 

(cf) Besides，unconstrained optimization with the penalized constraints in the 


objective function Sometimes works better than the constrained optimization 
routine“fmincon () 7”. 


Table P7.8 The Results of Penalty Methods Depending on the Initial Guess and 
Weighting Factor 


| Tesaning Pantm=4 0 | | Tesaning Pantm=4 0 | Point xo = [0.4 0.5] Xo 一 [0.2 4] 
[eaer| [minsearen steep minuno| mineon | 


1.21 1.34 1.34 1.34 
0.58 0.62 0.62 0.62 
区 到 一 一 


1.34 
0.62 
1.34 











0.00 0.00 0.00 0.00 0 00 0 00 0.00 0.00 22. 1 16. 4 








1.21 1.21 1.12 1.18 1.21 1.21 1.15 | 一 1.26 
0.58 0.58 0.76 0.64 0.58 0.58 0.71 1.70 


站 证 中 后 活 者 改 本 者 丰 区 浊 清 
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7.9 A Constrained Optimization on Location 


A company has three factories that are located at the points (一 16,4),，(6,3)， 
and (3, 一 9), respectively, in the xlx2-plane, and the numbers of deliveries 
to those factories are 9, 0, and 10 per month, respectively (Fig. P7.9). The 
company has a plan to build a new warehouse in ilts Site bounded by 


Ilxz1 一 二 十 lx 一 1 和 2 (P7.9.1) 


and is trying to minimize the monthly mileage of delivery trucks in deter- 
Imining the location of a new warehouse on the assumption that the distance 
between two points represents the driving distance. 


(a) What is the objective function that must be defined in the program 
“nm7p09 .m72 
(b) What is the statement defining the inequality constraint (P7.9.1)? 


(c) Complete and run the program “nm7p09.m” to get the opttmum location 
of the new warehouse. 


function [C,Cceq]l = fp _warehouse_c(x) 
C= Sum(abs(x - [11])) - 2; 
Ceq = []; 点 No equality constraint 


s%nm7p09.m to Solve the warehouse location problem 

f= "sqrt([Sum((Xx - [-16 4]).^2) sum((Xx - [6 5]).^2) sum((???3??3???).^2)]) ; 
fp_warehouse = inline([f '*[?;?;2?]']，Xx'); 

x0= [113;A= [lb= [];Aeq= []) beq= [1I= [us= [II 

XO fmincon(fp_warehouse,x0,A,b,Aedq,beq,1,uU，fp_warehouse_c ) 

















T T 
5 上 ， O 四 
〇 site factory B 
factory A 
0 上 引 
十 
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Figure P7.9 The site of a new warehouse and the locations of the factories. 


7.10 A Constrained Optimization on Ray Refraction 


Alight ray follows the path that takes the Shortest tme when it travels in 
the Space. We want to find the three angles 01,0, and 0 (measured between 
the array and the normal to the material Surface) of a Tray traveling from 
P=(0,0) to CO = (一 (di 十 十 中)) through a transparent material of 
thickness d> and index of refraction 7 as depicted in Fig. P7.10. Note the 
following things. 


(3) 


(b) 
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Since the Speed of light in the transparent material js v 一 c/P (cis the 
Speed of light in the free space), the traveling time to be minimized 
Can be expressed as 


dl 710 0 


Min F0,d,7, 世 ) 一 
站 CCcoSOl CCcoSoO ccos05 











(P7.10.1) 


The sum of the three horizontal distances traveled by the light ray mnust 
be 克 : 


克 
g(0,d,7, 工 ) 一 了 dtanb - 工 =0 (P7.10.2) 


The horizontal distance 也 and the index of refraction 7 are addition- 
ally included in the input argument lists of both the objective function 
丰 (0,d,7,) and the constraint function 8g(0,d,7, 志 ) regardless of 
whether or not they are used in each function. It is because the objective 
function and the constraint function of the MATITLAB routine“fmin- 
con()”must have the Same input arguments. 


Compose a program“nm7p10a.m”that Solves the above constrained 
Iminimization problem to find the three angles 01，6, and 03 for 克 一 
1.32,d 三 忆 =d 三 1[cml, anddifferent values of 世 一 0.0:0.3:0 and 
plots Sin(O1)/sin(0) and Sin(03)/Sin(02) versus 也 ， 

Compose a program “nm7p10b.m”that finds the three angles 01;,0， 
and 03 for 了 =3 cm, 矶 三 罗 三 必 三 1cm and diffterent values of 
17. 一 1:0.01:1.6 and plots sin(O1)/sin(6) and Sin(03)/Sin(02) versus 7， 







alight ray speed of light = C 
air 





玫 
全 
Speed of light = cm 
op transparent material 
with refraction index 














上 站 上 -| 


Figure P7.10 Refraction of alight ray at an air-glass interface. 


7.11 A Constrained Optimization on OFDM System 


In order to find the average modulation order xi for each user of an OFDM 
(orthogonal frequency division multiplex) System that has NMW(128) Subchan- 
nels to asSlgn to each of the four users in the environment of nolise power 
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No and the bit error rate (Probability of bit error) 已 , Seung-hee, a commu- 
nication System expert, formulated the following constrained minimization 
problem: 


Min F(x) = 0 D 字 2(erfe-K(P/2) 生 (P7.11.1) 


Subject to 


4 三 
gr) 一 人 本 二 N=0 (P7.11.2) 
with NM = 128，and awi: the data rate of each useT 


where erfc-1(x) is the inverse function of the complementary error function 
defined by Eq. (P4.9.3) and is installed as the MAILAB built-in function 
“erfcinv() .He defined the objective function and the constraint func- 
tion as below and Save them in the Mr-ftiles named“fp_bits1.m”and 
“fp_bits_c.m”. 


function y = fp_bits1(x,aN,Pe) 
NO=1;y= Sum((2.^x-1)*NO/3*2xerfcinv(Pe/2).^2.*a./X); 


function [CCcedq] = fp_bits_c(x,aN,Pe) 
C= []; cedqd = Sum(a./x) - Ni 





Compose aprogram that Solves the above constrained minimization problem 
(with No = 1 and 己 = 10-) to get the modulation order xi of each user 
for fve different sets of data rates 


4 = [32 32 32 32], [64 32 32 32], [128 32 32 32], [256 32 32 32]，and [512 32 32 32] 
and plots ali/xli(the number of subchannels assigned to user 1) versus al 
(the data rate of user 1). I you feel uneasy about the results obtained with 


your initial guesses, try with the initial guesses as follows for each set of 
data rates, reSpectively: 


X0 一 [0.3 0.3 0.3 0.3],[L11H,[L11H[2222]，and[4444] 


MATRICES AND 
EIGENVALUES 





In this chapter， we will look at the eigenvalue or characteristic value 入 and its 
corresponding eligenvector Or characteristic vector V of a matrix. 


8.1 EIGENVALUES AND EIGENVECTORS 


The eigenvalue or characteristic value and its corresponding eligenvector or char- 
acteristic vector of an NW x N matrix 4 are defined as a scalar 入 and a nonzero 
Vector V Satisfying 


4v=A 和 My 今 (4-XDv=0w 短 0) (8.1.1) 


where (和 ,Vy) is called an eigenpair and there are N eigenpairs for the N x N 
matrix 4. 
How do we get them? Noting that 


。 jn order for the above equation to hold for any nonzero vector V, the matriX 
[4 一 入 should be singular 一 that is, its determinant Should be zero (|4 一 


人 1|=0) 一 and 
。the determinant of the matrix [4 一 和 7] is apolynomial of degree N in termas 
of 入 ， 


we first must fnd the elgenvalue 和 Xi S by solving the So-called characteristic 
equation 
I4--AX1 = 和 ++aw_IX-L 二 .二 aa+ao=0 (8.1.2) 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang,，Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 人 儿 Sons, Inc. 
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and then Substitute the 和 S，one by one，into Eq. (8.1.1) to solve it for the 
elgenvector Vi S. This 1$s, however, not always So Simple, especially 让 Some root 
(eligenvalue) of Eq. (8.1.2) has multiplicity K > 1，Since we have to generate 大 
independent eligenvectors Satisfying Eq. (8.1.1) for such an eigenvalue. Still， we 
do not have to worry about this, thanks to the MAILAB built-in routine“eig()”， 
which finds us all the eligenvalues and their corresponding eligenvectors for a given 
matrix. How do we use it? All we need to do ls to define the matrix, Say 4, and 
type a Single statement into the MAILAB command window as follows. 


>>[V,Lambda]l = eig(A) %5e = elig(A) just for eigenvalues 
Let us take a look at the following example. 


卫 xXample 8.1. Eigenvalues/Eigenvectors of a MEatriX. 
Let us find the eligenvalues/eigenvectors of the matrix 


0 1 
4=|1 | (E8.1.1) 


First， we find its elgenvalues as 


一 人 1 
0 一 1 一 人 


+D=0， 和 =0) = 一 1 (E8.1.2) 


4-M=| | -=2+x=o 


and then，get the corresponding elgenvectors as 


0 1 ULL | V21 > 0 
| 


=0， Ti = 芭 沁 加 (E8.1.39) 
|11 1 2| lz+z| 10 
-am-[ 引 人-[“i]- 四 
1/V2 
U12 一 一 22， V2 一 | 一 加 | (上 8.1.3b) 


Where we have chosen uvll, vl2?，and uv2z> So that the norms of the eligenyvectors 
become one. 

Alternatively, we can use the MAILAB command “eig(A)”for fnding eigen- 
values/elgenvectors Or“roots(poly(A) )”just for finding eligenvalues as the 
roots of the characteristic equation as illustrated by the program“nm811.m”. 
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%nm811 to get the eigenvalues & eligenvectors of a _ matrix A， 
Clear 

A= [01;0 -1]; 

[V,L] = eig(A) %VvV = modal matrix composed of eigenvectors 


有 = diagonal matrix with eigenvalues on its diagonal 
elig(A) ，roots(poly(A)) %just for eigenvalues 
= V^ - 1*Axrv %qdiagonalize through Similarity transformation 
s%5 into a diagonal matrix having the eligenvalues on diagonal.， 





8.2 SIMILARITY TRANSFORMATION AND DIAGONALIZATION 


Premultiplying a matrix 4 by P-1 and postmultiplying itby P makes a similarity 
transformation 
4 一 Pr-I4P (8.2.1) 


Remark 8.1 tells us how a Similarity transformation affects the eigenval- 
Ues/eigenvectors. 


了 R 了 emark 8.1. Effect of Similarity Transformation on Eigenvalues/Eigenvectors 
1. The eigenvalues are not changed by a Similarity transformation . 


1P-L4P-AX1=1P-I4P- PITPI=1P-4-X1IPI=14 一 和 7 
(8.2.2) 
2. Substituting y = Pw into Eq. (8.1.1) yields 


4Av 一 入 vV， 4PW= 和 入 Pw= PXW， [P-I4P]w = 入 w 


This implies that the matrix P-14P obtained by a similarity transformation 
has WwW 一 P-Lv as its eigenvector 让 v is an eigenvector of the matrix 4. 


In order to understand the diagonalization of a matrix into a diagonal matrix 
(having its eigenvalues on the main diagonal) through a Similarity transformation， 
we have to know the following theorem: 


Theorem 8.1. Distinct Eigenvalues and Independent Eigenvectors. 

If the eigenvalues of a matrix 4 are all distinct 一 that is, different from each 
other 一 then the corresponding eigenvectors are Independent of each other and， 
consequently，the modal matrix composed of the eigenvectors as columns is 
nonsingular. 


Now, for an NM x Nmatrix 4 whose eligenvalues are all distinct, let us put all 
of the equations (8.1.1) for each eligenvalue-eigenvector pair together to write 


和 1 0 ， 0 
0 和 人 X :， 0 


4[vv :ywl=[vv vv | 47 = 三 YA 
区 0 ， | 
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Then，noting that the modal matrix V is nonsingular and invertible by Theo- 
rem 8.1，we can premultiply the above equation by the inverse modal matrix 
V-Lto get 

V-I4V = YIVA 三 人 (8.2.4) 


This implies that the modal matrix composed of the eligenvectors of a matrix 4 
ls the Similarity transformation matrix that can be used for converting the matrix 
4into a diagonal matrix having its eigenvalues on the main diagonal. Here is an 
example to jllustrate the diagonalization. 


卫 xample 8.2. Diagonalization Using the Modal Matrix. 
Consider the matrix given in the previous example. 


0 1 
4=|1 | (E8.2.1) 


We can use the eligenvectors (上 E8.1.3) (obtained in Example 8.1) to construct 
the modal matrix as 
1 1 


=mwm=|， -1/V5I 


and use this matrix to make a Similarity transformation of the matrix 4 as 


| 


1 1 0 -1/V2 0 0 
二 一 了 上 8.2.3 
5 5 可 上 二 2 
which is a diagonal matrix having the eligenvalues on lts main diagonal. 


This job can be performed by the last statement of the MAILAB program 
“nm811.m”， 


(E8.2.2) 


This diagonalization technique can be used to decouple an N-dimenslional 
vector differential equation So that it can be as easy to Solve as N independent 
Scalar differential equations. Here is an lllustration. 


卫 xample 8.3. Decoupling of a Vector Equation Through Diagonalization 


(a) For the linear time-invariant (LID state equation (0.3.3) 


2a0|_ [0 11Txao0| 0 
外 浊 近 民 | 人 | 市 [| 2s (1) (上 8.3.1) 
with 车 避 三 加 and xy( 人 = 王 1V1L>0 


X' (1) 一 4X() 十 BUDD) with the initial state X(0) and the input (1 
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we use the modal matrix obtained as (上 8.2.2) in Example 8.2 to make a Substi- 
tution of variable 


加 xz0]_ [1 UVV21Twr 
| 


which converts Eq. (E8.3.1) into 
Vw Gd) 一 4VwG) 十 Ba 人 CD) (上 38.3.3) 
We premultiply (E8.3.3) by V-1L to write it in a decoupled form as 


w()=V-I4VwGU) 二 VB OOD=Awd) 十 V-LBA CD with w(O) = YIX(O); 


攻 
wz | 10 =-1L||wz0) 0 -vV2|LI ”  L=wz =-V2xs0) 


(E8.3.4) 
[28]-[ 刁 [D]-[5 


Where there ls no _ correlation between the variables wl1(D) and w2(1). Then we 
can Solve these two equations Separately to have 


WwW1 CD 一 必 人 (人 with w1(0) = 0; 


Wi1() 一 V1(0) 三 TiG) 一 WwW1( 一 1 (上 8.3.5a) 


2， 
2 二 一 wo 人 0 - V2xs(D) with wz(0) = V2; 
2 
3W2(8) 一 VU2(0) = 一 2(8) 一 交 ， 
ua(0) V2 2002 


3 十 1] 8G 十 ]) 一 8 3 十 1] 
wz 人 一 V2(1 十 2e-0Dx(O) (E8.3.5b) 











Was) 三 


? 


and Substitute this into Eq. (E8.3.2) to get 


| 
2 人 | [0 -UL Lo -UV LV2(-1+2e-)] 
| 必 ( (E8.3.0) 


This ls the Same result as Eq. (6.3.10) obtained in Section 6.3.1. 
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(b) Suppose Eq. (E8.3.1) has no input term and So we can expect only the 
natural response resulting from the initial state，but no forced response 
caused by the input. 


X1 (1D) 本 0 1 X1(1) XI1(0) 加 1 
项 交 | 区 攻 9 WwWith 攻 洒 二 图 (上 8.3.7) 


We apply the diagonalization/decoupling method for this equation to get 
1 (D) 罗 入 1 0 1(T) 区 0 0 1(T) 
WwW |10 和 lz 10 =-1|1wz0) 


1 1 之 
mvocvo [28]-[ 本 中 -全 


Wi | we 网 
1 加 本 二 | 二 (上 8.3.8) 
X(1) (8&.3.2) YYw(1I) 一 [vi Vv?] 攻 一 WwW1(O)exuVi 再 WwW2(0)e“rv? 
下 1 1/V2 色 _ [2 2 
四 攻 | 医 罗 | Et | (上 8.3.9) 


As time goes by, this solution converges and So the continuous-time SyS- 
tem turns out to be Stable, thanks to the fact that all of the eigenvalues 
(0, 一 1) are distinct and not positive. 


卫 xample 8.4. Decoupling of a Vector Equation Through Diagonalization. 
Consider a discrete-time LII state equation 


xl[2 十 HH]| 10 1 xl1[z] 0 
1 


with 世人 一 加 andi[Im=1V7>0 (E8.4.1) 


In order to diagonalize this equation into a form Similar to Eq. (上 E8.3.4), we use 
MAILAB to find the eligenvalues/eigenvectors and the modal matrix composed 
of the eigenvectors and fnally, do the Similarity transformation. 


A= [01;0.20.1];B= [0; 2.2361]; 5 扣 Eq.(E8.4.1) 
[V,L] = eig(A) s%“V = modal matrix Composed of eigenvectors (E8.4.2) 
s%5L = diagonal matrix With eigenvalues on its diagonal 


Ap = V^-1*Ax*Vv %5diagonalize through Similarity transformation (E8.4.3) 
s%5 into a diagonal matrix having the eigenvalues on the diagonal 
Bp =V^-1*B 5 (E8.4.3) 
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Then，we get 
_ 0 1=-04 0 和 一 0.9283 ”一 0.8944 
oj 0 05 03714 -0.4472 
(上 E8.4.2) 
3 | -0.4 0 ad | 交 '6759 
4 一 YY 4VY= | 0 | and DB 一 V 也 一 医 :， 沁 (上 E8.4.3) 
So that we can write the diagonalized State equation as 
WII 十 上 | | -04 0 WwW1[7] 2.0759 
区 十 证 至 | 0 | 有 全 人 医 由] 
| 一 0.4wl[2] 十 2.0759 
四 | 0.5uwz[m] 一 | 0 


Without the input term on the right-hand Side of Eq. (E8.4.1)，we would have 
obtained 


7 十] 
人 | with w[0] 二 V-1x[0] 





WwW2[2 十 了 某 WwW2[7] 和 2 ww2[0] 
(上 8.4.93) 
xm] = ywDm] = [ww] 人 = ni[OMv 上 ul)gv (GE8.4.6) 
W2[0] 和 2 1 2 


As time goes by (ie., as 1 increasesj, this Solution converges and so the discrete- 
time System turns out to be Stable, thanks to the fact that the masgnitude of every 
elgenvalue (一 0.4, 0.9) ls less than one. 


了 R 了 emark 8.2. Physical Meaning of Eigenvalues and Eigenvectors 


1. As jllustrated by the above examples，we can use the modal matrix to 
decouple a set of differential equations So that they can be Solved one 
by one as a scalar differential equation in terms of a single variable and 
then put together to make the solution for the original vector differential 
equation , 


2. Through the above examples，we can feel the physical Significance of the 
elgenvalues/eigenvectors of the System matrix 4 in the state equation on jlts 
Solution. That is, the state of a linear time-invariant (LID System described 
by an NW-dimensional continuous-time (differential) state equation has NV 
modes {e 和 并 一 1,...，Nj,each of which converges/diverges 证 the sign of 
the corresponding eigenvalue ls negative/positive and proceeds Slowly as 
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the masgnitude of the eigenvalue ls close to zero. In the case of a discrete- 
time LII System described by an NW-dimensional difference state equation， 
its state has N modes { 人 7 并 1 三 1...，Nj,each of which converges/diverges 
it the magnitude of the corresponding eligenvalue is less/greater than one 
and proceeds Slowly as the masgnitude of the eigenvalue ls close to one. 
To summarize, the convergence property of a state X or the Stability of a 
linear-time invariant (LID System is determined by the eligenvalues of the 
System matrix 4. As illustrated by (上 E8.3.9) and (E8.4.6), the corresponding 
elgenvector determines the direction in which each mode proceeds in the 
N-dimensional state Space. 


8.3 POWER METHOD 


In this section，we will introduce the Scaled power method, the inverse power 
method and the shifted inverse power method, to find the eigenvalues of a 
given matrix. 


8.3.1 Scaled Power Method 


This method is used to find the eligenvalue of largest magnitude and is summarized 
in the following box. 


SCALED POWER METHOD 


Suppose all of the eigenvalues of an NV x N matrix 4 are distinct with the 
masnitudes 
|Ai| > |A2?| |X3| 过 :过 |AN| 


Then, the dominant eigenvalue 和 1 with the largest magnitude and its corre- 
Sponding eigenvector v1 can be obtained by starting with an initial vector X0 


that has Some nonzero component in the direction of vi and by repeating the 
following procedure: 

Divide the previous vector Xk by its largest component (in absolute value) 
for normalization (Scaling) and premultiply the normallized vector by the 
matrix 4. 


琉 
xl=A4- 一 > 和 NIV  with lxls=Maxfl| (8.3.1) 
xxellw 
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Piooj According to Theorem 8.1, the eligenvectors {Vo 7 一 1:NWjofanNxN 
matrix 4 whose eigenvalues are distinct are independent and thus can constitute 
a basis for an N-dimensional linear Space. Consequently，any initial vector X0 
can be expressed as a linear combination of the eigenvectors: 


X0 一 Q1V1 十 2V2 十 … :十 QNVN (8.3.2) 


Noting that 4v, = 和 mvn，we premultiply both Sides of this equation by 4 
to get 


4X0 三 QIA1IVI1 十 Q2 和 2V2 十 … :十 QN 和 ANVN 


入 2 入 N 
一 和 1 |oxivl 十 % 一 V2 十 .十 QN 一 VN 
入 1 入 1 


and repeat this multiplication over and over again to obtain 


XK 一 A4x0 
四 AN 
一 入 le 十 Q2 ( 疡 ) V2 十 …… 十 QN ( 乞 ) | Iaivl (8.3.3) 
1 1 


which will converge to an eigenvector V1 as long as al 和 0. Since we keep 
Scaling before multiplying at every iteration, the largest component of the limit 
Vector of the Sequence generated by Eq. (8.3.1) must be 和 1. 


XK V1 (8.LD) V1 


ES 5 (8.3.4) 
|xklle lvil。 vill。 








XKk+1 一 


Note that the scaling prevents the overfiow or underflow that would result from 
| 和 il > 1 or | 和 1| < 工 . 


了 R 了 emark 8.3. Convergence of Power Method 


1. In the light of Eq. (8.3.3)，the convergence Speed of the power method 
depends on how small the masgnitude ratio (| 和 2z|/| 和 il) of the second largest 
elgenvalue 和 over the largest eligenvalue 入 1 1S. 

2. We often use XI=[1 1 .1]as the initial vector. Note that 
让 it has no component in the direction of the eigenvector (V1) 
corresponding to the dominant eigenyalue 和 1 一 that is, xl 一 XosyliV/Vl 蝶 三 
0 in Eq. (8.3.2) 一 the iteration of the Scaled power method leads to the limit 
Showing the second largest magnitude eligenvalue 和 > and its corresponding 
elgenvector v2. But, if there is more than one largest (dominant) eligenvalue 
of equal magnitude, it does not converge to either of them， 
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8.3.2 Inverse Power Method 


The objective of this method is to find the (uniquely) smallest (magnitude) elgen- 
value 和 w by applying the scaled power method to the inverse matrix 4-!L and 
taking the inverse of the largest component of the limit, It works only in cases 
where the matrix 4 ls nonsingular and thus has no zero elgenvalue. Its idea is 
based on the equation 

4y=)y 一 4vY= 和 TV (8.3.5) 


obtained from multiplying both sides of Eq. (8.1.1) by 和 -14-1. This implies 
that the inverse matrix 4-!1 has the eigenvalues that are the reciprocals of the 
elgenvalues of the original matrix 4, sttl having the Same eligenvectors. 


] 


一 一 一 一 8.3.0 
> the largest eligenvalue of 4-1 4 ) 


8.3.3 Shifted Inverse Power Method 


In order to develop a method for finding the eigenvalue that is not neceSsSsarily 
of the largest or Smallest magnitude，we Subtract sy (S: a number that does not 
happen to equal any eigenvalue) from both sides of Eq. (8.1.1J) to write 


4y=AXV 一 [4-5S1y=( 和 人 一 9)V (8.3.7) 


Since this implies that (入 一 9) is the eligenvalue of [4 一 站, we apply the inverse 
power method for [4 一 7]to getits smallest magnitude eigenvalue (从 一 9) with 
minfl 和 一 | 王 1:WNWjandaddsytoitto obtain the eligenvalue of the original 
matrix 4 which is closest to the number 8. 


1 


二 EC 8.3.8 
the largest eigenvalue of [4 一 8 站 -1 本 ) 


8 


The prospect of this method is Supported by Gerschgorin's disk theorem， 
which is summarized in the box below. But, this method is not applicable to the 
matrix that has more than one eligenvalue of the same masgnitude. 


Theorem 8.2. Gerschgorin's Disk Theorem. 

Every eligenvalue of a square matrix 4 belongs to at least one of the disks 
(in the complex plane) with center wmwm (one of the diagonal elements of 4) and 
radius 


my 一 >》， |amz|(the sum of all the elements in the row except the diagonal element) 
妖 天 11 
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Moreover，each of the disks contains at least one eigenvalue of the 
matrix 4. 

The power method introduced in Section 8.3.1 is cast into the routine 
“eig_power()” The MAILAB program “nm831.m” uses it to perform the power 
method, the inverse power method and the shifted inverse power method for 
fnding the eligenvalues of a_ matrix and compares the results with that of the 
MATLAB built-in routine“eig()”for cross-check. 


function [Lambda,v] = eig_power(A,X,EPS,MaxIter) 

%5 The power method to find the largest eigenvalue (Lambda) and 

s% the corresponding eigenvector (v) of a matrix A. 

if nargin < 4，MaxIter = 100; end 点 maximum number of iterations 

if nargin < 3，EPS = 1e-8; end %5 difference between Successive Values 
Size(A,2) 

if nargin<2,，Xx= [1:N]; ends% the initial Vector 


lambda = 0 

for k = 1:MaxIter 
x1 = xj 1lLambdal = lambda; 
Xx = A*x/norm(x,inf); %5Eq.(8.3.4) 
[xm,m] = max(abs(x)); 
Lambda = Xx(m); 5 the component With Largest magnitude(absolute value) 
if norm(x1 - x) < EPS & abs(1Lambda1-1Lambda) < EPS，break;i end 

end 

if k == MaxIter，disp('Warning: you may have to increase MaxIter ' ); end 
= X/norm(x) ; 


s%nm831 

%App1ly the power method to find the 1Largest/smallest/medium eigenvalue 
Clear 

A= [201;0 -20;102]; 

X= [123]';s%x= [111]'; 各 With different initial Vector 

EPS = 1e-8; MaxIter = 100; 

%the largest eigenvalue and its corresponding eigenvector 

Lambda_max,vV] = eig_power(A,Xx,EPS,MaxIter) 

%the Smallest eigenvalue and its _ corresponding eigenvector 

Lambda,v] = eig_power(A^ - 1,Xx,EPS,MaxIter) 

Lambda_min = 1/1Lambda， Vv s%Eq.(8.3.6) 

%eigenvalue nearest to a number and its _ corresponding eigenvector 

S= -3j AsI= (A- S*eye(Size(A)))^ - 1 

1Lambda,Vv] = eig_power(AsI,Xx,EPS,MaxIter) 

Lambda = 1/1Lambda+s %5Eq.(8.3.8) 

fprintf('Eigenvalue Closest to %54.2f = %58.4f\nwith eigenvector ' ,S, Lambdal) 
V 











V,LAMBDA] = eig(A) %modal matrix Composed of eigenvectors 


8.4 JACOBIMETHOD 


This method finds us all the eigenvalues of a real Symmetric matrix. Its idea is 
based on the following theorem. 
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Theorem 8.3. Symmetric Diagonalization Theorem. 

All of the eigenvalues of an NM x N symmetric matrix 4 are of real value and 
its eligenvectors form an orthonormal basis of an NW-dimensional linear Space. 
Consequently，we can Imake an orthonormal modal matrix W composed of the 
eigenvectors Such that V7VY = 7 VY- = V7 and use the modal matrix to make 
the Similarity transformation of 4，which yields a diagonal matrix having the 
elgenvalues on its main diagonal: 


VIT4V = YI4V = 入 (8.4.1) 


Now, in order to understand the Jacobl method，we define the pqg-rotation 
Imatrix as 


pu column du column 
1 0 0 0 0 
0 1 0 0 0 
Roy(O) 三 0 0 cosO 一 0 | pa row (8.4.2) 
0 0 sing0 cosO 0 | gu row 
0 0 0 0 1 


Since this is an orthonormal matrix whose row/column vectors are orthogonal 
and normalized 


多 E | 
民生 (8.4.3) 


premultiplying/postmultiplying a matrix 4 by R2 /Ra makes a Similarity trans- 
formation 


40) 一 R2 ARpa (8.4.4) 


Noting that the Similarity transformation does not change the eigenvalues (Re- 
mark 8.1), any matrix resulting from repeating the Same operations SucceSsSsively 


4D = 民 人 ADRO = RORAD RAR RDRO (8.4.5) 


has the Same eigenvalues. Moreover, if it ls a diagonal matrix, it will have all 
the eigenvalues on its main diagonal, and the matrix multiplied on the right of 
the matrix 4 is the modal matrix V 


了 一 尺 .…. 必 DRW (8.4.0) 


as manifested by matching this equation with Eq. (8.4.1). 


JACOBI METHOD 


function [LAMBDA,V,ermsg]l = eig_Jacobi(A,EPS,MaxIter) 

%Jacobi method finds the eigenvalues/eigenvectors of Symmetric matrix A 

if nargin < 3，MaxIter = 100; end 

if nargin < 2，EPS = 1e-8; end 

N = Size(A,2) 

LAMBDA =[]; V = []; 

form= 1:N 
if norm(A 


(A(m:Nm) - A(mm:N) ) > EPS 
error( 


( 
asSymmetric matrix!l ); 


max(abs(A(m;m + 1:N))); 


[Amm,p] = max(Amn)) 9q9= pr+Qp); 
if Amm < EPS*sum(abs(diag(LAMBDA) ) ) ，break ; 
if abs(A(p,p)-A(q,q) )<EPS 
S2 1; S=1/sqrt(2);)c= Si 
CxCcj SS = S*Sj 


2*A(p,q)/(A(p,p) - 
1/Sqrt(1 + 
C = Sqrt((1 + c2)/2) 
cC = C*Ccj SS = S*xSsj 
end 
LAMBDA = Ai 
6 = A(p,:)*c+Aq,:)*sj 
LAMBDA( : LAMBDA(p，: ) ' ; 
LANBDA(G -A(p,:)*S + A(q:)*Cij 
LAMBDA( : LAMBDA(q，:) '; 
LAMBDA( 0; LAMBDA(q， 
LAMBDA( A(p;,p)*cc +A( 
LAMBDA( = A(p,p)*SsSs +A( 


V(:，[p 这 V(:，[P 9q])*[C -SisS C] 
end 
LAMBDA = diag(diag(LAMBDA)); %for purification 


%nm841 app1lies the Jacobi method 
s%5 to find all the eigenvalues/eigenvectors of a Symmetric matrix A. 
clear 
= [201;0 -20;102]; 
EPS = 1e-8;j MaxIter =100; 
[L,V] = eig_Jacobi(A,EPS,MaxIter) 
disp('Using eig() ) 
[V,LAMBDA] = eig(A) %modal matrix composed of eigenvectors 
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What ls left for us to think about is how to make this matrix (8.4.5) diag- 
onal. Noting that the Similarity transformation (8.4.4) changes only the pth 


IOwWS/columns and the gth rowSs/columns as 





田光 
Upg 二 Up 二 0qp(C 一 9 ) 十 (aa4 一 0pp)3C 


1 
一 aqp COS 20 十 本 (0aa 一 Qpp)Sin20 (8.4.7a) 
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um 一 Up 一 GapnC 十 ons forthe pa rowy/column with 头 pqg (8.4.7b) 


un 一 Unp 三 一 apng 十 aonc forthe dg row/column with 妹夫 DPI9 (8.4.7c) 
Upp 一 appC 十 ao782 十 204p08cC 一 appC 十 ao082 十 ap Sin20 (8.4.7d) 
Ua9 一 pp3 十 dogC2 一 247p08C 一 app27 十 ao7C2 一 Qpr Sin20 (8.4.7e) 


(C 一 cos0,8 一 SinO0) 
we make the (P,9g) element vpr and the (9g, P) element vop Zero 
up 一 uvp 一 0 (8.4.8) 


by choosing the angle 0 of the rotation matrix 尺 or(O) in Such a way that 








亢 Sin20 2ap0 20 1 1 
an < 一 三 一 一， coS 20 = 一 -一 一 一 一， 
cos20 amp 一 Goa Sec20 ”V1+tan220 
sin20 一 tan20cos20 (8.4.9) 
in20 
cosO 一 Vcos20 = V(l 二 cos20)/2， sing 一 
2cosO 


and computing the other associated elements according to Eqs. (8.4.7b--e). 

There are a couple of things to note. First, in order to make the matrix closer 
to a diagonal one at each iteration，we Should identify the row number and the 
column number of the largest offt-diagonal element as P and qg, respectively, and 
Zero-out the (P,qg) element. Second，we can hope that the masgnitudes of the 
other elements in the pth,qgth row/column affected by this transformation process 
don't get larger, Since Eqs. (8.4.7b) and (8.4.7c) implies 


Un 十 un 一 (Gorc 十 aonS) 十 (一 0png8 十 aonc) 三 六 十 0 (8.4.10) 

This so-called Jacobi method is cast into the routine“eig Jacobi()” The 
MAILAB program“nm841.m” uses it to fnd the eligenvalues/eigenvectors of a 
matrix and compares the result with that of using the MAILAB built-in routine 
“eig()”for cross-check. The result we may expect ls as follows. Interested 
readers are welcome to run the program“nm841.m”. 


2 0 1 3 0 0 
4=|0 -2 0| 一 RO4R3=|0 -2 0|= 和 
1 0 2 0 0 1 

1/V2 0 -1/V2 
WwWith RI3 一 0 1 0 一 了 


l/V2 0 1/vV2 
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8.5 PHYSICAL MEANING OF EIGENVALUES/EIGENVECTORS 


According to Theorem 8.3 (Symmetric Diagonalization Theorem), introduced in 
the previous Section, the elgenvectors {v 1 一 1:NWjofanNxANSymmetric 
matrix 4 constitute an orthonormal basis for an NM-dimensional linear Space. 


1 for 7 一 7 


0 for 7 夫 忆 人 


V7VY = 1 六 全 号 | 


Consequently, any NW-dimensional vector X can be expressed as a linear combi- 
nation of these elgenvectorS. 


N 
X 一 QIV1I 十 o2V2 十 :…' 十 QNVN 一 》， QnVn (8.5.2) 


ET 


Thus, the eigenvectors are called the principal axes of matrix 4, and the squared 
norm of a vector is the sum of the squares of the components (ar S) along the 
Principal axis. 


N 7 7/ N N  N N 
人 (二 en 旧 | ES 
类 三 二 


首 二 下 州 二 下 二 二 二 1 
5.3) 
Premultiplying Eq. (8.3.2) by the matrix 4 and using Eq. (8.1.1) yields 
N 
A4X 一 和 1Q1V1 十 和 202V2 十 … :十 入 VONVN 一 >》， 和 nzOmVn (8.5.4) 
1 一 1 


This Shows that premultiplying a vector X by matrix 4 has the Same effect as 
multiplying each principal component on of X along the direction of eligenvector 
Vvn by the associated elgenvalue 和 人 m. Therefore, the solution of a homogeneous 
discrete-time state equation 


AN 
xK 十 1) 一 4x( with x(0) = 六 ovV，， (8.5.5) 
洒 三 直 
can be written as 区 
x(o = 》 和 anvn (8.5.6) 
17 一 1 


whpich was jllustrated by Eq. (E38.4.0) in Example 8.4. On the other hand, as illus- 
trated by (上 8.3.9) in Example 8.3(b), the solution of a homogeneous continuous- 
time state equation 


AN 
xD 一 4x() withn x(O) = 》 anw (8.5.7) 


入 = 
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can be written as 


AN 
x(f) 一 》， eartQV， (8.5.8) 
1 一 1 


Equations (8.3.6) and (8.5.8) imply that the eligenvalues of the System matrix 
characterize the Principal modes of the System described by the state equations， 
That 1，the eligenvalues determine not only whether the System is Stable or 
not 一 that 1S$，whether the System State converges to an equilibrium state OF 
diverges 一 but also how fast the System state proceeds along the direction of 
each eligenvector. More Specifically, in the case of a discrete-time System,， the 
absolute values of all the eligenvalues must be less than one for Stability and 
the Smaller the absolute value of an eligenvalue (less than one) is, the faster the 
Corresponding mode converges. In the case of a_ continuous-time System, the real 
parts of all the eigenvalues must be negative for Stability and the Smaller a neg- 
ative eligenvalue is, the faster the corresponding mode converges. The difference 
among the eigenvalues determines how Stiff the System is (See Section 0.5.4)， 
This meaning of eigenvalues/eigenvectors ls very important in dynamic Systems. 

Now, in order to figure out the meaning of eligenvalues/eigenvectors in Static 
Systems，we define the mean vector and the covariance matrix of the vectors 
{X0), xO2，...,XA)} representing 天 points in a two-dimensional space called the 
X1X2 plane as 


开 开 
1 1 
加 天 二 》 文 忆 ， (对 元 》 [x@ 一 ma][xw 一 mx]7 (8.5.9) 
k=1 人-1 


where the mean vector ITepresents the center of the points and the covariance 
Imatrix describes how dispersedly the points are distributed. Let us think about 
the geometrical meaning of diagonalizing the covariance matrix Cr. As a Simple 
example, Suppose we have four points 


XGO) 一 医 虽 XGO) 一 | 细 XG) 一 邑 XG) 一 | (8.5.10) 


for which the mean vector mi, the covariance matrix C, and its modal matrix 


are 
] 2.3 2 1 1 1 
m=| | | | VY=[vi o= 蕊 | -1 | 


Then，we can diagonalize the covariance matrix as 


本 和 限 天 二 鸭 裔 本 本 本 
| 
引 |。 人 交 ， (8.5.12) 
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wphich has the eligenvalues on its main diagonal. On the other hand, if we trans- 
form the four point vectors by using the modal matrix as 


y=V G 一 mo) (8.5.13) 
then the new four point vectors are 


vO -= | | 全 医 * 人 人 区 “人 二 三 | | 


-3/V2 [=-3/V2 | 3/V2 [3/vV2 
(8.5.14) 
for which the mean vector my and the covariance matrix Care 
my = V7m, -mv) = 加 ， cy=Yrcey = 路 加 一 和 
(8.5.19) 


The original four points and the new points corresponding to them are depicted 
in Fig. 8.1, which shows that the eigenvectors of the covariance matrix for a set of 
point vectors represents the principal axes of the distribution and its eligenvalues 
are related with the lengths of the distribution along the Principal axes. The 
difference among the eigenvalues determines how oblong the overall shape of 
the distribution is. 

Before closing this section，we may think about the meaning of the deter- 
minant of a matrix composed of two two-dimensional vectors and three three- 
dimensional vectors. 





























3 T 
乡 | 二 
1 上 二 
0 
_1 上 ] 
衬 人 二 ] 
yG@) y(D) 
3 上 | 
-4 -2 0 2 4 


Figure 8.1  Eigenvalues/eigenvectors of a covariance matrixX. 
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EFirst, let us consider a 2 x 2 matrix composed of two two-dimensional vectors 
XGD and XO). 
光 全 “这 
和 一 [xD xOO] 一 | 1 (8.5.16) 
X21  X22 
Conclusively, the absolute value of the determinant of this matriX 


det(X) 一 | 和 | 一 XY11X22 一 X12X21 (8.3.17) 


edquals the area of the parallelogram having the two vectors as its two nelighboring 
Slides. In order to certify this fact， let us make a clockwise rotation of the two 
vectors by the phase angle of xD) 


-gb 二 一 tanr1l ( 芋 ) (8.5.18) 


必 11 


So that the new vector y\ 山 corresponding to xD) becomes aligned with the xl1-axis 
(See Fig. 8.2). For this purpose, we multiply our matrix X by the rotation matriX 
defined by Eq. (8.4.2) 


RCo)=| | | | 人 | (8.5.19) 
Sin( 一 01) cosOl j 测 一 X21 XI11 
to get 
7 = R(-0)X 一 es | 2 | 区 | (8.5.20a) 
人 一 21 XI | | xz 7222 
1 X2 十 2 X11X12 十 X21X 
上 vvO] 1 十 2X21 11X12 十 X21X22 
人 人 0 一 X12X21 十 XI11X22 
X11 十 221 


The parallelograms having the original vectors and the new vectors as their two 
neighboring Sides are depicted in Fig. 8.2, where the areas of the parallelograms 
turn out to be equal to the absolute values of the determinants of the matrices XX 
and 了 as follows: 

Area of the parallelograms 

一 Length of the bottom side x Height of the parallelogram 

一 (X1 component of yO)) x (xz component of yCO)) 一 y11y22 一 det(7) 


2 2 本 
二 2 及 二 22721 十 xl11722 = det(X) (8.5.21T) 


/2 2 /2 2 
X11 十 X21 X11 十 X21 
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X2 

全 一 一 一 一 一 一 一 一 一 一 
个 y@) 1 
1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

1 

0 一 6| | 

(TD) 
了 上 和 X1 
V 作 1 十 和 
(al A parallelogram (b) The rotated parallelogram 


Figure 8.2 Geometrical meaning of a determinant. 


On extension of this result into a three-dimensional Situation，the absolute 
value of the determinant of a 3 x 3 matrix composed of three three-dimensional 
vectors XC,X2，and XG) equals the volume of the parallelepiped having the 


three vectors as its three edges. 


XI1 XI2  X13 
det(X) = |XI=|x0 xxO  XxG|=| ml xz 223| 三 XOD XXO .XI 
X31  X32 33 


3) 


(8.5.22) 


8.6 EIGENVALUE EQUATIONS 


In this section，we consider a System of ordinary differential equations that can 


be formulated as an eligenvalue problem. 
For the undamped mass-Sspring System depicted in Fig. 8.3, the displacements 
X1() and x2(1) of the two masses 111 and 112 are described by the following 


System of differential equations: 


区 4 攻 十 妃 )/zl | 区 让 


2 人 | 一 D/1112 jpD/112 | | xz2(1) 
X1(0) X1(0) 
with | and 区 二 
X (1) 一 一 4X(D) with x(0) and x“ (0) (8.6.1) 


Lettheeigenpairs (eigenvalue-eligenvectors) ofthe matrix 4 be (人 一 02， vn) with 


Avh 一 ozvn (8.6.2) 
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spring Spring 
constant constant 
Xi 人 Xe 人 
上 1 人 六 2 


LT 


Figure 8.3 An undamped mass-spring system. 

















Noting that the solution of Eq. (8.3.7) can be written as Eq. (8.3.8) in terms of 
the eligenvectors of the System matrix，we write the solution of Eq. (8.6.1) as 


网 
x0D)=》 ww=[v mm] 局 = ywd) (8.6.3) 


7 一 1 


and Substitute this into Eq. (8.6.1) to have 


52 _ 


2 风 2 
由 OOw=-4》 wuwDw 全 一 》 wozw (8.6.4) 
了 7 一 1 


由 (一 -own() forz=12 (8.6.5) 
The solution of thlis equation ils 


六 和 吉大 测定 罗 交 有 交 册 和 机 用 二 证 六 站 避让 莹 让 
CO1 
(8.0.0) 
where the initial value of w() = [wi(D) wz(]7 can be obtained via Eq. (8.6.3) 
from that of X(f) as 


(8.6.3) (8.4.0 


w(0) V-IxX(O) V7X(0)， w'(0) = V7Tx'(0) (8.6.7) 


Finally，we Substitute Eq. (8.6.6) into Eq. (8.6.3) to _ obtain the solution of 
Eq. (8.6.1). 
PROBLEMS 


8.1 Symmetric Tridiagonal TIoeplitz Matrix 


Consider the following N x N symmetric tridiagonal TIoeplitz matrix as 


4 D 0 0 0 
DP 0 2 0 0 
0 D 4 0 0 
(P8.1.1) 
0 0 0 4 2 
0 0 0 D 4a 
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(a) Verify that the eigenvalues and eigenvectors of this matrix are as follows， 
with N = 3 for convenience. 


写 站 8668 (5 formn 二 1to N (P8.1.2) 


2 5 刀 元 . 2711 元 7 元 史 
V， 一 ,| 一 一 一 | Sn | 一 一 一 Sn | 一 一 一 | Sn (P8.1.3) 
N+l N+l Nd+l N+l 


Letting N =3,4a 王 2, and2D 王 1 fndthe eigenvalues/eigenvectors of 
the above matrix by using (P8.1.2,3) and by using the MAILAB routine 
“eig Jacobi()”or “eig()”for cross-check. 





(b 


se 


8.2 Circulant Matrix 
Consider the following N x WN circulant matrix as 
7 (0) ]P(V 一 1) PN-2 : AGO) 
7(]) 7(0) ]P(VN 一 1 :… 72) 
/2) 1 jz(0) 7G3) 82.0) 


1(VN 一 ID) PNVN-2 HAN-3 70) 


(a) Vertify that the eigenvalues and eligenvectors of this matrix are as follows， 

with N = 4 for convenience. 
入 一](0) 十 PN 一 De 二 ON 一 2)ej22  (P8.2.2) 

二 人 2 于 户 (])e72rC 一 DAX 
V， 二 [1 ej2ra/N ej2r2n/W ..，ej2r(N-DWN]7 (P8.2.3) 
for7 一 0toN 一 1 

(b) Letting N =4,7(0) 一 2,1(03) 一 7) 三 1and72) = 一 0,findthe eligen- 
values/eigenvectors of the above matrix by using (P8.2.2,3) and by using 


the MAILAB routine“eig_ Jacobi()”or “eig()” Do they agree7 Do 
they satisfy Eq. (8.1.1)? 


8.3 Solving a Vector Differential Equation by Decoupling: Diagonalization， 
Consider the following two-dimensional vector differential equation (State 
equation) as 


xl (| | 0 11Tx(OD 0 
区 辣 全 | 必 十 [| 21) (P8.3.D) 
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which was Solved by using Laplace transform in Problem P6.1. In this prob- 
lem，we Solve it again by the decoupling method through diagonalization of 
the System Imatrix. 


(a) Show that the eligenvalues and eligenvectors of the System matrix are 3S 
follows. 


入 1 一 一 1， 入 2? 一 一 2; 呈 汪 | 全 -| (P8.3.2) 


(b) Show that the diagonalization of the above vector differential equation 
using the modal matrix V = [yl  v2 ] yields the following equation: 


WOD | | 一 1 0 | | w1(0) ] 
-Do ms 
| 一 WOD 十 必 人 OO .| wo | | 2 
攻 人 六: 网 7 人 阅 国 
(c) Show that these equations can be Solved individually by using Laplace 


transform technique to yield the following solution，which is the Same 
as Eq. (P6.1.2) obtained in Problem P6.1(a). 








1 1 
用 网 = ee 人) (P8.3.4a) 
一 1/2 17/2 1 2 
mpO= 一 一 OO=-70+e Du (834b) 
1 十 E 二 人 亲生 
[= [+ 人] oo eeaa 


8.4 Householder Method and QR Factorization 


This method can zero-out Several elements in a column vector at each iter- 
ation and make any N x N matrix a (lower) triangular matrix in (V 一 了 J) 
jiterations， 


(a) Householder Reflection (Fig. P8.4) 
Show that the transformation matrix by which we can multiply a vector 
X to generate another vector y having the Same norm is 


妃 = [1 一 2ww] 


太一 了 
民 一 yl 


5 1 
with w 一 = 人 一 史 ,和 = 怪 一 列 ， 民 | = 他 | 全 34.1) 


and that this is an orthonormal symmetric matrix such that 瓦 7 克 = 
妃 厅 = 7 五 -= 万. Note the following facts. 
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x 一 y 


Xx-y=cw w=EK-y -让 
2 





|xll=llyi 


Figure P8.4 Householder reflection. 


(iD y 一 X 一 区 一 y) 人 X 一 CW (P8.4.2a) 
二 WwWTwW (2 莽 ) 1and lxll = yl (P8.4.2b) 
(这 ) m 一 G 十 y)/2 一 X 一 (c/2)wW (P8.4.2c) 


(ivy) The mean vector m of X and yis orthogonal to the difference vector 
w= 区 一 J)/c. 
Thus we have 


w7GX 一 (c/2)w) = 0; w7Xx-(c/2)wIrw=wrx-(c/2) =0 
(P8.4.3) 
This gives an expression for c 一 | 肛 一 yll> as 


c= 民 一 yl = 2w*x (P8.4.4) 
We can Substitute this into (P8.4.2a) to get the desired result. 
y 一 X 一 CW 一 X 一 2ww7x=[1 一 2ww7]x= 囊 x (P8.4.9) 


On the other hand, the Householder transform matrix ls an orthog- 
onal matrix，Since 
媚 7 刀 = BE =[L1--2ww7]U7 一 2ww] 
= 了 一 4ww7 二 4wwTww7 
= 了 一 4ww7 十 4ww7 =7 (P8.4.0) 
(b) Householder Transform 
In order to Show that the Householder matrix can be used to zero-out 


Some part of a vector, let us find the kth Householder matrix ZE trans- 
forming any vector 


X 一 [2 1 MK LN ] (P8.4.7) 
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into 
y 一 [2 1L 一 8 0 .…: 0] (P8.4.8) 


Where 8k ls fxed in such a way that the normas of these two vectors are 
the Same: 





(P8.4.9) 
First, we find the difference vector of unit norm as 
1 
wk 三 一 习 一 了 ) 
C 
1 
三 沁 [0 … 0 了 欢 十 8 MK xxXN] GP8.4.10) 


with 


< 三民 一 yl = VCxe 十 80D2 十 X8 1 十 :十 2 (P8.4.11) 


Then, one more thing we should do is to substitute this difference vector 
into Eq. (P8.4.1). 
及 =[ 一 2wkewj (P8.4.12) 


Complete the following routine “Householder()”by permuting the 
statements and try it with KK 王 1，2,，3,，and 4 for a four-dimensional 
vector generated by the MAILAB command rand(5,1) to check if it 
Works fine. 


>> X= rand(5,1)，for k = 1:4，householder(x,k)*x，end 


function H = Householder(x,k) 

%Householder transform to zero out tail part Starting from kK + 1 
H eye(N) - 2xwxwWw'; %5Householder matrIix 

JIength(x) ; 

Zeros(N,1); 

) =(xX(k) + g)/cj wk+ 1:N) = X(kK + 1:N)/ci %Eq.(P8.4.10) 

= Sum(Xx(k + 1:N). > 2); 

sqrt((x(k) + g)^2 + tmp); #%Eq.(P8.4.11) 

Sdqrt(x(k)^2 + tmp); %5Eq.(P8.4.9) 


K 
p 





QR Factorization Using Householder Transform 

We can use Householder transform to zero out the part under the main 
diagonal of each column of an N x N matrix 4 successively and then 
make it a lower triangular matrix Rin (CN 一 JU iterations. The necessary 
operations are collectively written as 


已 IE 2 4A= 尺 (P8.4.13) 
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which implies that 
4=[Ew_ iv …]-IR= 五 5 忆 DR 
一 刀 .…PNv PN IR=OR (P8.4.14) 
where the product of all the Householder matrices 
@=:…BEv-2Ev-l (P8.4.15) 
turns out to be not only symmetric, but also orthogonal like each 及: 
070 = [ 珈 EvaoPw 而 Eva2Bw 
= 万 EBP li 一 7 


This suggests a OCR factorization method that is cast into the following 
Ioutine“qr_my()”. You can try it for a nonsingular 3 x 3 matrix gener- 
ated by the MAILAB command rand(3) and compare the result with 
that of the MATITLAB built-in routine“qr() 7” 


function [Q,R] = qr_my(A) 

%5QR factorization 

N = Size(A,1); R= AijQ= eye(N); 
fork= 1:N -1 


Householder(R(:,k),k); 
H*R; %5Eq. (P8.4.13) 
Qx*H; %5Eq. (P8.4.15) 





8.S Hessenberg Form Using Householder Transform 


function [Hs;,HH] = Hessenberg(A) 

s%Transform into an almost Upper triangular matrix 

s%5 having only zeros below lower subdiagonal 
Size(A,1); Hs = Aij HH = eye(N); %HHxAxHH'” = HSs 


1:N - 2 
Householder(Hs(:,k)， ) ; 
Hs = Hx*HsxH; HH = Hx*HH; 





We can make use of Householder transform (introduced in Problem 8.4) to 
Zero-out the elements below the lower Subdiagonal of a matrix So that 
becomes an upper Hessenbersg form which is almost upper-triangular matrix. 
Complete the above routine“Hessenberg()”by flling in the second input 
argument of the routine “Householder()”and try it for a 3 x3 matrix 
generated by the MAILAB command rand(5) to check if it works. 
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QR Factorization of Hessenberg Form Using the Givens Rotation 


We can make use of the Givens rotation to get the QR factorization of Hessen- 
berg form by the procedure implemented in the following routine 
“qr_Hessenberg()” where each element on the lower Subdiagonal is zeroed 
out at each iteration. Generatea4 x 4random matrixAby the MAILAB com- 
mand rand(4), transform it into a Hessenberg form Hs by using the routine 
“Hessenberg()”and try this routine“qr_Hessenberg()”for the matrix of 
Hessenberg form. Check the validity by seeing 过 norm(Hs-Qx*R) s 0 or not. 


function [Q,R] = qr_Hessenberg(Hs ) 

%QR factorization of Hessenberg form by Givens rotation 
N = Size(Hs,1); 

Q = eye(N); R = 

for kK 1 


;jyY=R(k+lik); mm= sqrt(xx*x + yxy); 





Diagonalization by Using QR Factorization to Find Eligenvalues 


You will see that a real Symmetric matrix A can be diagonallized into a 
diagonal matrix having the eigenvalues on its diagonal 夺 we Tepeat the 
Similarity transformation by using the orthogonal matrix Q obtained from the 
QR factorization. For this purpose, take the following steps. 


function [eigs,A] = elig_QR(A,kmax ) 

%Find eligenvalues by using QR factorization 

if nargin < 2，kmax = 200; end 

for k = 1:Kkmax 
[Q,R] = qr(A); %A = Qx*R; R =Q'*A =Q^-1x*A 
A = Rx*rQjij %5A = Q” - 1*AxQ 

end 

eligs = diag(A) ; 


function [eligs,A] = elig_QR_Hs(A,kmax ) 

%Find eigenvalues by using QR factorization via Hesenberg 
if nargin < 2，kmax = 200; end 

Hs = hessenberg(A) ; 

for k = 1:Kmax 


[Q,R] = qr_hessenberg(Hs); %5Hs = Q*R; R = Q'*HSs 
HSs = RxQji %Hs = Q”^ - 1*HSxQ 

end 

eigs = diag(Hs) ; 
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(a) Make the above IToutine“eig_QR()”that uses the MATITLAB built-in 
routine“qr()”and then apply itto a4 x4random symmetric matrix A 
generated by the following MAILAB statements， 


>> A= rand(4))A=A+A' 


(b 


\ 必 


Make the above routine“eig_QR_Hs()”that transforms a given matriX 
into a Hessenberg form by using the routine“Hessenberg()”(appeared 
in Problem 8.3) and then repetitively makes the QR factorization by 
using the routine“qr_Hessenberg()”(appeared in Problem 8.0) and 
the Similarity transformation by the orthogonal matrix Q _ until the matrixX 
becomes diagonal. Apply it to the 4 x 4 random symmetric matrix A 
generated in (a) and compare the result with those obtained in (a) and 
by using the MAILAB built-in routine“eig()”for cross-check. 


8.8 DifferentialDifference Equation，State Equation, and Eigenvalue 


As mentioned in Section 0.5.3, a high-order scalar differential equation Such 
as 


XGOG() 十 aoxzO(D 十 ax CD 十 aox( 一 2OD) (P8.8.1) 


can be transformed into a first-order vector differential equation，called a 
State equation，as 


X1 (1) 0 1 0 X1(D) 0 
oo0| -| 0 0 11aeo|1+lolan ss82a 
X3 (1) 一 40 一 0 一 02 X3 (1 1 
X1(T) 
x() 一 [1 0 0]| zx (P8.8.2b) 
X3 (1 


The characteristic equation of the differential equation (P8.8.1) is 
S3 十 0D87 十 al18 十 ao 一 0 (P8.8.3) 


and its roots are called the characterlstic roots. 


(a) What is the relationship between these characteristic Toots and the elgen- 
values of the System matrix 4 of the above state equation (P8.8.2)?7 To 
anSwer this question，write the equation | 和 7 一 4|=0to solve for the 
elgenvalues of 4, and show that it is equivalent to Eq. (P8.8.3). Io extend 
yoOur experience or just for practice, you can try the Symbolic computation 
of MAILAB by running the following program“nm8p08a.m”. 
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%nm8p08a 
Syms a0 al a2 S 
A=[010;001;-a0 -al -a2]; 5%5(P8.8.2al) 


det(S*xeye(Size(A))- A) s%characteristic polLlynomial 


ch_eq = poly(A) %5or，equivalent1Ly 





Let the input (GD in the state equation (P8.8.2) be dependent on the state 
as 
2U(I) 一 KXI) 一 [Koxl() KKIx2z()  K2X3(1)] (P8.8.4) 


Then, the state equation can be written as 


X1 (1D) 0 1 0 X1(1) 
X2 (1) | 一 0 0 1 X2(1) (P8.8.3) 
X3 (1) 天 0 一 00 开 I1 一 0 天 2 一 02 X3(1) 

Ithe parameters of the original system matrix are ao 一 1 al = 一 2, and 


ad2 一 3，what are the values of the gain matrix 开 一 [Ko K1 K2?] you 
will fx So that the virtual System matrix in the state equation (P8.8.3) 
has the eigenvalues of 和 人 = 一 1 一 2, and 一 3? Note that the character- 
istic_ equation of the System whose behavior is described by the state 
equation (P8.8.3) ls 





5 十 (oa 一 Ko) 上 (一 K)s+a 一 Ko=0 (P8.8.60) 
and the equation having the roots of 入 三 一 1, 一 2, and 一 3 is 
GC+HDG+2G6+3)= +6 +T1l15+6=0 (P8.8.7) 


8.9 A Homogeneous Differential Equation 一 An Eigenvalue Equation 

Consider the undamped mass-Spring System depicted in Fig. 8.3，where the 
masses and the Spring constants are 1111 一 1, 12 一 1[kg] and 司 一 9, 心 一 10 
[Nm]，respectively. Complete the following program “nm8p09.m”whose 
objective jls to solve the Second-order differential equation (8.6.1) with the 
initial conditions [xl1(0), xz(0), x1(0), xz(0)] = [1, 一 0.5,0,0] for the time 
interval [0,10] in two waySs 一 that 1$，by using the ODE-solver “ode45()” 
(Section 6.35.1) and by using the eigenvalue method (Section 8.0) and plot 
the two solutions. Run the completed program to obtain the solution graphs 
for xl1(I) and x2 (1). 

(cf) Note that the Second-order vector differential equation (8.6.1) can be written as 


the following state equation: 


Xi)| |0 7 X(1) 
区 ]-[2 5。 em 
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snm8p09.m Solve a Set of differential eqs， (a state equation) 
Clear，CLf 
global A 
df = df861 ; 
k1=5;k2= 10;jml1= 1; m2= 1; sthe spring constants and the masses 
[(kl1 + k2)/ml -k2/m1; -k2/m2 k2/m2]; NA = Size(A,2) 
to0 = 0j tf =??;j X0 =[? ???? ? ?]; 5% initial/final time，initial values 
[t4,Xx4] = ode45(df,[to tf],xo); 
[V,LAMBDA] = eig(A); 5 modal matrix Composed of eigenvectors 
WO = x0(1:NA)*V; WwW10 = Xx0(NA+1:end)*V; 5 Eq.(8.6.8) 
omega = 2232223232332323232323232322?32322323 
forn= 1:NA55Eq.(8.6-7) 
omegan=omega(n) ; 
w(:n) = [cos(omega nji*t4) Sin(omega nxt4)]*[woln);w1o0(n)/Vomega nj]; 
end 
xE = WxV.' ;55Eq.(8.6.3) 
for n = 1:NA 
Subplot(311 + n)，plot(t4,x4(:,n)，b'，t4,xE(:,n)，r) 
end 


了 


function dx = df861(t,X) 

global A 

NA = Size(A,2) 

if length(x) ~= 2*NA，error('Some dimension problem'); end 
dx = [zeros(NA) eye(NA)j -A zeros(NA)]*x(:); 

if Size(x,2) > 1，dx = dx. ; end 





PARTIAL DIFFERENTIAL 
EQUATIONS 





What is apartial differential equation (PDE)? It is a class of differential equations 
invyolving more than one Independent variable. In this chapter, we consider a gen- 
eral Second-order PDE in two independent variables x and y， which is written as 


8 入 02 927 0 837 
4 y) 一 一 十 有 By) 一 一 十 CO y) 一 一 一 ,VD 一 ,一 9.0.1 
(x 二 十 BCx 7 到 四 十 CG 力 575 了 (4 7 了 池 ) (9.0.1) 


for xzo 和 XY<Xf,y0 SS<y1 
with the boundary conditions given by 


U(CC， y0) 一 Dy0C)， MG yYF) 三 DrCx)， 
(9.0.2) 
U(x0, y) 一 Drxo(y)， and VC yy) 一 Drer(O7) 


These PDESs are classified into three groupS: 


Elliptic PDE: 让 8 一 44C <0 
Parabolic PDE: 让 玉 一 44C =0 
Hyperbolic PDE: 让 8 一 44C >0 
These three types of PDE are associated with equilibrium states, diffusion states， 


and oscillating Systems, respectively. We will Study some numerical methods for 
Solving these PDESs, since their analytical solutions are usually difficult to find. 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang, Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 儿 Sons, Inc. 
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9.1 ELLIPTIC PDE 


As an example，we will deal with a special type of elliptic equation called 
Helmholtz's equation，which is written as 


V200, 十 gCy)G y) 一 


382 3920 
二 一 一 十 SC, yy 三 Je y) (9.1.1) 
0OX 0y 
over a domain 万 = {(x, y)lxo 和 xx 和 XYj yo 三 》<yr 让 with some boundary con- 


ditions of 
U(Xo, y) 三 Dro(y)， LO y) 三 Drr(y)， 


(9.1.2) 
UCc, y0) 一 DoCX)， and CO,yr) 一 DrGCo) 


(cf) Equation (9.1.1) is called Poisson's equation 过 gx,y) 三 0anditis called Laplace's 
equation 过 g(x,y) 三 0 and jx,y) =0. 


To apply the difference method, we divide the domain into Msections, each 
of length Ax = (Xr 一 X0)/AM along the x-axis and into Msections，each of 
length Ay = (yyr 一 yo0)/AM along the y-axis, respectively, and then replace the 
Second derivatives by the three-point central difference approximation (5.3.1) 





827(x， 1 一 2720 iJ-1 
| 
DX2 站 这 AxX2 
琶 (9.1.3a 
| 四) 一 Ge 攻 ) (9.1.3b) 
37 7 Ji Ay” 





So that, for every interior point (xj yi) with1<7z< 和 < 一 1andl1 和 7 <AM 一 |， 
we obtain the finite difference equation 


下 闭 1 一 2 十 三 OH 一 2 十 只 一 1 本 
和 (9.1.4) 


where 
有 二 2 六 太 )， 太太 太 )，and 8 一 8 六 ) 


These equations can Somehow be arranged into a System of Simultaneous 
equations with respect to the (MI 一 DC 一 D variables {211 212，. .21M 1 
22,1，. 121 -1 NM -11， UNM,-12，...， WAM 11}， but it seems to be 
messy to work with and we may be really in trouble as M，and MI ，become 
large. A Simpler way is to use the iterative methods introduced in Section 2.3. 
To do So, we first need to shape the equations and the boundary conditions into 
the following form: 


开放 ( 本 1 十 帮 产 D) 雨 OH 十 厦门 十 Poy(8i70 一 大 让 (9.1.5a) 
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Ui0 一 Dr0(0i)， Ms 一 DerOi)，Uoj 一 Do 一 Droci) (9.1.5b) 
where 


Ay” Ax? Ax2Ay 


和 
了 


How do we initialize this algorithmy HI we have no Priori knowledge about the 
Solution, it ls reasonable to take the average value of the boundary values as the 
initial values of 局. 

The objective of the MAILAB routine“poisson.m” is to Solve the above 
equation , 


function [u,x,y] = poisson(f,g,bx0,bxf,by0,byf,D,Mx,My,tol,MaxIter) 
%Solve UXx+UuUyy+g(xy)u=TfT(xy) 
Over the region D = [x0,xf,y0,yf] = {(x,y) |x0 <= X <= xf，y0 <= yY <= yf} 
With the boundary Conditions : 
uU(x0,y) = bx0(y)，U(xf,y) = bxf(y) 
uU(x,y0) = by0(x)，U(xyf) = byf(x) 
=# of Subintervals along Xx axis 
My =# of subintervals along y axis 
tol : error tolerance 
MaxIter: the maximum # of iterations 
=D(1); xf=D(2);yo=D(3);yf=D(4); 
= (xf - x0)/Mxi xx= xo+ [0:Mx]*dxi 
(yYf - y0)/Myi yY=yo+ [Oo:My] *dyi 
1; MY1= My + 1; 
s%Boundary conditions 
form= 1:My1，u(m,[1 Mx1])=[bxo(y(m)) bxf(y(m))]; end s%left/right Side 
forn= 1:Mx1，u([1 My1l,n) = [by0(x(n)); byf(x(n))]; end %bottom/top 
s%initialize as the average of boundary values 
Sum_of _bv = Sum(Sum([uU(2:My,[1 Mx1]) uU([1 My1],2:Mx) ])); 
uU(2:My,2:Mx) = Sum_of_bv/(2*(MXx + My - 2)); 
for = 1:My 
for j = 1:Mx 
F(i,j) = Tf(x(] i)); G(ij) = g(x(j),y(i)) 
end 
end 
dx2 = dxx*dxj dy2 = dy*dy;j dxy2 = 2*(dx2 + dy2); 
= dx2/1dxy2j ry = dy2/1dxy2; Frxy = Frxx*dy2; 
for itr = 1:MaxIter 
for j = 2:Mx 
for = 2:My 
u(i,j) = ry*(u(ij + 1)+uij -1)) +rx*uir+ 1 j)+uiz - 1))，， 
+ Prxy*(G(i,j)*u(ij)- F(I,j)); s#%Eq.(9.1.5a) 
end 
end 
if itr>1&max(max(abs(uU - U0))) < to1l，break;i end 
u0 = Ui 
end 


s%Solve_poisson in Example 9.1 

f = inline('0'，x','y')) g= inline('0'，x'，Yy')); 

x0=0;j xf= 4;jMG= 20;) y0=0;jyf= 4; My = 20; 

bx0 = inline('exp(y) - cos(y)  ，y'); (E9.1: 
bx inline('exp(y)*cos(4) - exp(4)*cos(y)  ，y')j; 5(E9.1. 
by0 inline('cos(x) - exp(x) ，Xx'); 全 下 
byf = inline('exp(4)*cos(X) - exp(x)*cos(4)' ，Xx'); 5(E9.1. 
D= [xoxfyoyflij MaxIter = 500; tol = 1e-4; 

[U,x,y] = poisson(f,g,bxo,bxf,byo,byf,D,Mx,My,tol,MaxIter) ; 
Clf，mesh(x,y,U)，axis([0404 -100 100]) 


( 
p 
( 
( 
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卫 xXample 9.1. Laplace's Equation 一 Steady-9tate TIemperature Distribution , 
Consider Laplace's equation 


32 y) 92x0c,y) 本 


V7 ， 
4 0x? 07” 


0 for0<XY<4,0 和 二》 芝 4 


(上 E9.1.1) 
with the boundary conditions 


1U(0, y) 一 e” 一 cos y， U(4,y7) 一 ercos4 一 ecosy (上 9.1.2) 


2U(X, 0) 一 coOSX 一 e7 WU(z, 4) 一 edcosxz 一 ercos4 (上 9.1.3) 


What we will get from solving this equation ls (xz, y)，which Supposedly 
describes the temperature distribution over a Square plate having each side 4 
units long (Fig. 9.1). We made the MAILAB program“solve_poisson.m”in 
order to use the routine“poisson()”to Solve Laplace'`s equation given above 
and run this program to obtain the result shown in Fig. 9.2. 


Now, let us conslder the So-called Neumann boundary conditions described as 


OU(CXY,，y) 


可 三 D，(y) forx = Xo (the leftrside boundary) (9.1.7) 
克 


X 一 X0 





Dirichlet-type boundary condition (function value fixed) 


TFT 





JyMy 

















本 


























Neumann-type boundary condition (derivative fixed) 


Figure 9.1 The grid for elliptic equations with Dirichlet/Neumann-type boundary condition. 
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0 0 


Figure 9.2 Temperature distribution over a plate 一 Example 9.1. 


Replacing the first derivative on the leftrside boundary (x = Xo) by its three-point 
central difference approximation (3.1.8) 


1 一 胃 一 1 


入 Du (yi)， 丰 1 久 大 1 一 2D4 (yiD)Ax fori =12,... ,1M 一 1 


(9.1.8) 
and then Substituting this constraint into Eq. (9.1.Sa) at the boundary points，we 
have 


1i0 三 六 (1 十 有 -1) 十 CH10 十 三 -10) 十 Foy(8i000 一 上 态 0) 
三 户 (G1 十 研一 22005)Ax) 十 产 C6+10 十 中 -10) 十 Fry(8i0U0 一 态 0) 
二 27 有 1 十 产 (HL0 十 -10) 十 关 y(85000 一 万 o 一 20 (07)AAz) 
forz 三 12,...， 1 一 1 (9.1.9) 


HI the boundary condition on the lower Side boundary (> = yo) ls also of 
Neumann type, then we need to wiite Similar edquations for / 一 1 2,... ,Ah 一 1 


Mo 三 方 (4o. 六 1 十 10 六 D 二 27 十 my(8ojUo 一 思 ) 一 20 CAAy) 
(9.1.10) 
and additionally for the left-lower corner point (X0，yo)， 


10.0 一 20yU0.1 十 mu1L0) 十 my(800400 一 轴 o 一 2(0u 00)MAX 十 20y(x0)AAy)) 
(9.1.11) 
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9.2 PARABOLIC PDE 


An example of a parabolic PDE is a one-dimensional heat equation describing 
the temperature distribution (xz,1t) (x is position, 1 is tme) as 


322(c,D 9xCc, 人 四 
4 一 一 -一 
0X2? 0 





for0<x<XfF，0<1L 和 了 (9.2.1) 


In order for this equation to be Solvable, the boundary conditions (0, 1) 一 
po() 久 xCxj bt 三 Drr() as well as the initial condition (xz, 0) 三 10() Should 
be provided. 


9.2.1 The Explicit Forward Euler Method 


To apply the finite difference method,， we divide the spatial domain [0, xr] into 
AM sections, each of lengh Ax = Xr/AM, and divide the time domain [0, 7] into 
N Segments，each of duration Al = T/N, and then replace the second partial 
derivative on the leftrhand side and the first partial derivative on the right-hand 
Side of the above equation (9.2.1) by the central difference approximation (3.3.1) 
and the forward difference approximation (5.1.4), respectively, So that we have 


开 大 大 
4 一 26 十 奏 -1 人 册 


一 9.2.2 
AX2 Ar 4 ) 


This can be cast into the following algorithm, called the explicit forward Euler 
method，which is to be Solved iteratively: 


2 一 72 十 DT 一 2 withr 一 4 (9.2.3) 
for 7 一 12,...,hM 一 1 
To find the stability condition of this algorithm，we Substitute a trial solution 
1 一 )Ke7ir 人 COP is any nonzero integeT) (9.2.4) 
into Eq. (9.2.3) to get 
入 一 re 人 十 em 人 十 (27)=1 一 27(1-cos(r/P)) (9.2.5) 


Since we must have | 入 | < 1 for nondivergence, the Stability condition turns out 


to be 
和 (9.2.6) 
7 一 二 2 
Ax2 一 2 
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function [u,xt] = heat_exp(a,xf,T,ito,bxo,bxf,M,N) 
%5Solve a UXxXx=uUt for0<=XxX<= xf，0<= 七 <= 工 
Initial Condition: U(x;,0) = it0O(X) 
Boundary Condition: U(O,t) = bxo(lt)，UuU(xf,t) = bxf(t) 
M=#of subintervals along Xx axis 
N=#of subintervals along 七 axis 
X= [0O:M] *dxi 
tt= [0O:N]*dt; 
，U(I1) it0O(x(i)); end 
，uUC1M+1)n) = [bxolt(n)); bxf(t(n))]; end 
2x*r 


1 
1 


1;k) + uli-1,k)) + rixu(ik); %Eq.(9.2.3) 





This implies that as we decrease the spatial interval Ax for better accuracy，we 
must also decrease the time step Al at the cost of more computations in order 
not to lose the Stability. 

The MAILAB routine“heat_exp()”has been composed to implement this 
algorithm. 


9.2.2 The Implicit Backward Euler Method 


In this section， we consider another algorithm called the implicit backward Euler 

method，which comes out from substituting the backward difference approxima- 

tion (9.1.0) for the first partial derivative on the right-hand side of Eq. (9.2.1) as 
隐 2 十 o1 节 加 tt 


一 9.2.7 
AxX2 Ai ) 


Ai 
一 r 克 十 (十 2 站 大 一 7 本 夏 with 一 45 (9.2.8) 
forz 一 12,...,hM 一 1 


HI the values of 区 and 2 at both end points are given from the Dirichlet 
type of boundary condition, then the above equation will be cast into a System 
of Simultaneous equations: 


12 几 十 ra 
| -rr 1+2r 一 ， 0 0 | D0 2 
0 -rr 1+27 . 0 0 区 曙 1 
0 0 0 十 27 人 有 2 人 7 
0 0 0 一 ”下 二 好 ij 站 守则 
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How about the case where the values of 0/axzl*=o = DI(D) at one end are 
giveny In that case, we approximate this Neumann type of boundary condition by 


站 9.2.10) 
2Axr ) 他 
and mix it up with one more equation associated with the unknown variable 1 
一 rz 十 (1 十 27) 丰 一 7 一 2 (9.2.11) 
to get 
(十 27 夏 一 27 相 二 2 一 27 的 (OAx (9.2.12) 


We augment Eq. (9.2.9) with this to write 
1 一 27rp0(CJ)AX 


1 十 27 一 27 0 0 0 0 1 
-rr 1+2r 一 0 0 0 中 人 
0 一 太 1 十 27 一 太 0 0 夏 1 
0 0 一 太 1 十 27 0 0 夏 一 2 
0 0 0 1 十 27 一 7 二 7 
0 0 0 一 1 十 27 多 ut 十 7 
(9.2.13) 


Equations Such as Eq. (9.2.9) or (9.2.13) are really nice in the Sense that they 
can be Solved very efficiently by exploiting their tridiagonal structures and are 
guaranteed to be stable owing to their diagonal dominancy. The unconditional 
Stability of Eq. (9.2.9) can be Shown by Substituting Eq. (9.2.4) into Eq. (9.2.8): 


1 


一 jzr/P 1 27) 一 JP 一 1 入 ， 人 
莹 十 (1 十 2r) 一 re / 1 十 27(1 一 cos(r/P)) 


IX| < 1 


(9.2.14) 
The following routine“heat_imp()”implements this algorithm to Solve the 
PDE (9.2.1) with the ordinary (Dirichlettype of) boundary condition viaEq. (9.2.9). 


function [u,xit] = heat_imp(a,xf,T,it0,bx0o,bxf,M,N) 
%5Solve a U XXX=Ut for0<=X<= xf， 0<= tt<=T 
% Initial Condition: U(x,0) = it0(x) 
Boundary Condition: uU(0O,t) = bxolt)，U(xf,t) = bxf(t) 
# of Subintervals along Xx axis 
# of Subintervals along tt axiS 
X= [0:M] *dx; 
t= [0:N]*dt; 


1:M+ 1，uU(i1) itO(x(Ii)); end 


1:N+1，uU(C1M+ 1],n) = [bxolt(n)); bxf(t(n))]; end 


FF= axdt/dx/dx; F2 
for II= 1:M-1 
A(Iiyi) r2; %5Eq.(9.2.9) 
A(I -1，i) = -rijAii- 1) = -ri end 


2 全 


[rxu(1,k); zeros(M - 3,1);，rxu(M+ 1，k)] + U(2:M,k - 
:M,k) = trid(A,b); 
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9.2.3 The Crank-Nicholson Method 


Here, let us go back to see Eq. (9.2.7) and try to improve the implicit backward 
Euler method. The difference approximation on the left-hand Side is taken at 
time point K，while the difference approximation on the Tight-hand Slide is taken 
at the midpoint between time KK and 上 一 1 if weregard it as the central differ- 
ence approximation with time step A1/2. Doesn't this seem to be inconsistent? 
How about taking the difference approximation of both sides at the same time 
point 一 Say, the midpoint between 上 十 1 andK 一 for balance? In order to do so， 
we take the average of the central difference approximations of the left-hand Side 
at the two points K 十 1 andK, yielding 





， = 二 一 (09.2.15) 


人 +H1 1 二 1 大 D 天 大 人 1 大 
4 1 一 2 十 只 2 十 01 一 20 十 奏 -1 加 
Ai 


which leads to the so-called Crank-Nicholson method: 


一 Fa 十 2(] 十 六 24T1 一 7 三 Fa 十 2(] 一 7 六 2 十 72A (9.2.10) 
Ai 


with 7 一 4 一 一 
AxX2 


With the DirichleVNeumann type of boundary condition on Xo/XxM，IesSpec- 
tively, this can be cast into the following tridiagonal System of equations. 


2(1 十 7 一 0 0 0 2 
ER 0 0 2 
0 一 / 2(1 二 7r) . 0 0 友 时 
0 0 0 .20 十 门 一 2 
[ 0 0 0 -一 2 20 让 | 本 
2(1 一 门 太 0 0 0 在 7C 人 tr 十 2 
7 2(1 一 门 天 . 0 0 故 
0 太 2(1 一 门 . 0 0 夏 
三 十 
0 0 0 .2( 一 门 太 天 7 0 
0 0 0 27 2(1 一 六 2 27(00y 十 D 十 DOD) 


(9.2.17) 
This System of equations can also be Solved very efficiently, and its uncondi- 
tional Stability can be Shown by Substituting Eq. (9.2.4) into Eq. (9.2.10): 


2 和 (1 十 7r(1L 一 cos(T/P))) 三 2(1 一 r( 一 cos(r/P)))， 
1 一 7r(1 一 cos(r/P)) 


人 | 人 和 1 (9.2.18) 
1 十 7r(1 一 cos(r/P)) 


This algorithm is cast into the following MAILAB routine“heat_CN()”. 
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function [u,x,t] = heat_ CN(a,xf,T,Iito0,bxo,bxf,M,N) 
%5Solve aUXxXx=Ut for0<=Xx<= xf，0<= Tt<= 工 
Initial Condition: U(x;，0) = it0O(Xx) 
Boundary Condition: uU(O,t) = bxo(t)，UuU(xf,t) = bxf(t) 
M=#of subintervals along X axis 
=# of subintervals along tt axis 
xf/M; X= [0:M] *dxi 
T/N; = [0:N]xdti 
i= 1:M+1，u(il) = ito(x(i)); end 
= 1:N+1，u(I1M+ 1l,n) = [bxolt(n)); bxf(t(n))]; end 
= axdt/dx/dx; 
-Pr);r2=2xr(1+ Pi 
- 1 
r1;i #%Eq,.(9.2.17) 
- 1,I) = -rrj AiI- 1) = -ri end 


[rxu(1,k); zeros(M - 3,1); Prxu(M 区 六 下， < 六 
+ Prx(ull:M- 1k - 1) +U(3:M 1,k - 1)) + r2xu(2:M,k - 1); 
uU(2:M,k) = trid(A,b); %5Eq.(9.2.17) 





卫 xample 9.2. One-Dimensional Parabolic PDE: Heat Flow Equation. 
Consider the parabolic PDE 


827(0x 人 OU(x ,1) 


for0<x< 二 10<1L<0.1 (上 9.2.1) 
0X? 91 


with the initial condition and the boundary conditions 
2U(x, 0) 一 Sin TrX， 2(0,1) 一 0， 2U(,1ID) 三 0 (上 9.2.2) 


We made the MAILAB program “solve_heat.m” in order to use the routines 


?9 566 


“heat_exp()”“heat_imp()” and“heat_CN()”in solving this edquation and ran 
this program to obtain the results Shown in Fig. 9.3. Note that with the Spatial 
interval Ax =XYr1/M = 1/20 and the time step At = T/N = 0.1/100 = 0.001， 
we have 

Ai 0.001 


一 4 一 一 1] 一 0.4 上 9.2. 
本 | 





which satisfies the stability condition ( < 1/2) (9.2.6) and all of the three meth- 
ods lead to reasonably fair results with a relative error of about 0.013. But， 
让 we decrease the Spatial interval to Ax = 1/23 for better resolution，we have 
六 = 一 0.023, violating the Stability condition and the explicit forward Euler method 
(heat_exp()”) blows up because of instability as Shown in Fig. 9.3a，while 
the implicit backward Euler method (heat_imp()”) and the Crank-Nicholson 
method (heat_CN()”) work qujlte well as shown in Figs. 9.3b,c. Now,， with the 
Spatial interval Ax = 1/23 and the time step Al = 0.1/120, the explicit method 
as well as the other ones works well with a relative error less than 0.001 in return 
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0 0 0 0 
(a) The explicit method (b) The implicit method 


1 0.1 





0 0 
(c) The Crank-Nicholson method 
Figure 9.3 ”Results of various algorithms for a one-dimensional parabolic PDE: heat equation. 


for Somewhat (30%) more computations, despite that 7 = 0.3208 doesn't strictly 
Satisfy the Stability condition . 

This implies that the condition ( < 1/2) for stability of the explicit forward 
Euler method is not a_ necessary one，but only a sufficient one. Besides,，if 站 
Converges, its accuracy may be better than that of the Implicit backward Euler 
method, but generally no _ better than that of the Crank-Nicholson method. 


s%5SO1LVve_heat 

a= 1; s%the parameter of (E9.2.1) 

it0 = inline('Sin(pi*xx)"，X'); s%initial condition 

bx0 = inline('0');j bxf = inline('0'); sboundary condition 
xf=1;jM=25;) T=0.1;N= 100; %rr= 0.625 

sanalytical Solution 

uUo = inline(' Sin(pi*Xx)*exp(-pixpixt) ，Xx'， 七 '); 

[u1,x,t] = heat_exp(a,xf,T,it0o,bx0o,bxf,M,N) ; 
figure(1)，clLf，mesh(t,x，uU1) 

[u2,x,t] = heat_imp(a,xf,T,it0,bxo,bxf,M,N); s%sconverge unconditionally 
figure(2)，clLf，mesh(t,Xx,U2) 

[u3,x,t] = heat_CN(a,xf,T,it0o,bxo,bxf,M;N); %converge unconditional1ly 
figure(3)，clLlf，mesh(t,x,U3) 

MN = MxN; 

Uo = Uo(xt); auUo = abs(Uo)+epSs;i %values of true analytical Solution 
#%How far from the analytical SolLlution? 

err1 = norm((u1-Uo)./auUo)/VMN 

err2 = norm((u2-Uo)./auUo)/MN 

err3 norm((u3-Uo) ./auo)/VMN 
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9.2.4 Two-Dimensional Parabolic PDE 
Another example of a parabolic PDE is a two-dimensional heat equation describ- 
ing the temperature distribution U(xz,，y, 四 )((xz,y) is position, ft ls time) as 
392 32 yi 9 y,1 
1 UL(CC，》 HOC 用 
0x2 097y” 01 


forxo<XY<Xf yo<y<yr0<LI < 了 


(9.2.19) 


In order for this equation to be Solvable, we should be provided with the boundary 
conditions 


MU(X0， y， 1) 一 px0(y， 四 )， 2U(CX， y,1) 一 Duer(y， 四)， 
UCC, yo 四 一 Dyo01， and MUGyP1 一 Dyrt 


as well as the initial condition (xy, 0) 一 10(X，y). 

We replace the firstrorder time derivative on the rightrhand Side by the three- 
point central difference at the midpoint (tr1 十 灰 )/2 just as with the Crank 一 
Nicholson method. We also replace one of the second-order derivatives, Uxrxr and 
WUyy,， by the three-point central difference approximation (5.3.1) at time 灰 and the 
other at time 丸 +l1, yielding 


大 K 二 1 K+1 人 +1 人 1 大 
(一 十 玲 产 二 十 三 一 -和 一 共 / 


AxX2 A5 Ai 
(9.2.20) 
which seemas to be attractive, Since it can be formulated into a tridiagonal System 
of equations with respect to 必 生 记 世 0 and xi .But why do we treat zax 


and yy with discrimination 一 that is, evaluate one at time 太 and the other at time 
如 +H1in a fxed manner? In an alternate manner，we write the difference equation 
for the next time point 灰 +1 as 


人 
5 8 Ai 
(9.2.21) 
This formulation, proposed by Peaceman and Rachford [P-1j, is referred to as the 
alternating direction Implicit (ADD method and can be cast into the following 
algorithm: 


一 六 (tt1L 十 十 三 放 十 (] 十 27y)xr 一 三 7 1 十 大 和 十 (] 一 27x)2 


1 一 1 ,7 
for1<7<AM 一 1 (9.2.22a) 
三 的 Ci 十 号 逢 站 直人 示 27z 一 二 六 Gui 十 7 十 到 六 下 十 = 277D07 


for1<i<M 一 1 (9.2.22b) 
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with 
产 一 4AHAX， 方 =4ALMAY， 
Ax=(Cf 一 X0/Mr，Ay=(Or 一 yo ， AL=T/N 


The objective of the following MAILAB routine “heat2_ADI()”is to imple- 
ment this algorithm for Solving a two-dimensional heat equation (9.2.19). 


function [u,x,yt] = heat2_ADI(a,D,T,ixy0,bxyt,Mx,My,N) 
%Solve Ut= cuXxx+uUyy) for D(1) <=X<= D(2)，D(3) <=y<= D(4)，0 <= tt<=T 
Initial Condition: U(x,y;0) = Ixy0(x,y) 
Boundary Condition: U(x,yt) = bxyt(x,y,t) for (x,y)cB 
Mx/My = # of Subintervals along Xx/yY axis 
N=#of subintervals along t axis 
= (D(2) - D(1))/Mx; x = D(1)+[O0:Mx]*dx; 
(D(4) - D(3))/My; y = D(3)+[O:My] '*dy; 
T/N; tt= [0:N]*dt; 
s%Initialization 
for j =1:Mx+1 
for ITI= 1:My +1 
u(i,j) = ixyO(x(j),y(i)); 
end 
end 
Frx = ax*dt/(dx*dx); rx1 = 1 + 2*rXxj Frx2 = 
ry = axdt/(dy*dy); ry1=1+ 2xryij ry2 
下 9 户 村 ,MX s%Eq: (9.252228) 
Ay(j,j) = ry1; 
ij>1，Ay(j - 1】j -ryi Ay(j jj-1) 
end 
for II= 1:MY -1 .22b ) 
Ax(I,i) = Fx1; 
ifi>1，Ax(i - j -Fx;i Ax(i,I - 
end 
for k 13:N 
uU_1 Ui 七 = kxdti 
for ITIL= 1:My +15%Boundary condition 
u(i,1) = feval(bxyt,Xx(1)，y(i) st); 
uU(I,Mx+1) = feval(bxyt,Xx(Mx+1),y(i)，t); 
end 
for j=1:Mx+1 
uU(1,j) = feval(bxyt;,Xx(j)，y(1):t); 
uU(My+1,j) = feval(bxyt,x(j),y(My + 1):t); 
end 
if mod(k,2) = 
for TI = 2: 
jj = 2:Mx; 
bx = [ry*u(Ii,1) zeros(1,Mx - 3) ryx*u(i;My + 1)] 
+rxx(u_1(i-1,jj)+ ut1G+1)jj)) + rx2x*u 1(i)jj); 
u(i,jj) = trid(Ay,bx')'; s%5Eq.(9.2.22a) 
end 
el1se 
for j = 2:Mx 
ii = 2:My; 
by [rxx*u(1,j); zeros(My-3,1); rx*u(Mx + 1,j)] 
+ ryx*(u_1(ii,j-1) + ut(iij + 1)) + ry2xu_1(ii)j); 
u(ii,j) = trid(Ax,by); %5Eq.(9.2.22b) 
end 
end 
end 
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1 








0 0 


Figure 9.4 A solution for a two dimensional parabolic PDE obtained using “heat2_ADI()” 
(Example 9.3). 


卫 xample 9.3. A Parabolic PDE: Two-Dimensional Temperature Diffusion. 
Consider a two-dimensional parabolic PDE 


9200 yy 有 92200c yy,1 DC yt 
10-4 U(X，y 2 2C0 沁 人 9 六 
0x2 0y 01 
for 0<Y<4，0<y><4，0<1f<3000 


(上 9.3.1) 


with the initial conditions and boundary conditions 
U(xz,y,0) 一 0 for 二 0 (上 9.3.2a) 
WU, y, 加 一 e"cosx 一 ecogyforx 一 0,x 一 4 一 0,y 一 4 (上 E9.3.2b) 


We made the following MAILAB program“solve_heat2.m” in order to use 
the routine“heat2_ADI()”to solve this equation and ran this program to get the 
result Shown in Fig. 9.4 at the final time. 


%5SO1LVe_heat2 
Clear，CLf 
= 1e-4; 
it0 = inline('0'，Xx'，y'); 5%5(E9.3.2a) 
X) - 


bxyt = inline('exp(y)*cos( exp(X)*coSs(y)"，X'，y'，t'); 5%5(E9.3.2b) 
D= [0404];jT=5000jMx= 40j My = 40jN= 50) 

[u,x,yt] = heat2_ADI(a,D,T,ito,bxyt,Mx,My,N) ; 

mesh(x,y)U) 





9.3 HYPERBOLIC PDE 


An example of a hyperbolic PDE is a one-dimensional wave equation for the 
amplitude function (xz, 1 站 (xz is position, ft ls tme) as 


02x(0cD 32 人 ce 


A 
0X2? 012 


for0<x<Xrf， 0 三 1 苹 荆 (9.3.1) 
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In order for this equation to be Solvable, the boundary conditions (0, 1) 一 
po() and Cj,1t) 三 Drr() as well as the initial conditions (xzx,0) = 10(x) and 
90/8ll;=o(x, 0) = 芭 Cc) should be provided. 


9.3.1 The Explicit Central Difference Method 


In the same way as with the parabolic PDESs,，we replace the second derivatives 
on both slides of Eq. (9.3.1) by their three-point central difference approximation 
(3.3.1) as 


人 并 大 K 十 1 大 K 一 1 
Zi 1 2117 十 2 1 一 217 十 妃 


人 A 一 
AX2 人 A 友 


必 太 全 
with Ax 一 一 ,Alfl 一 一 (9.3.2) 
MI AN 


which leads to the explicit central difference method: 


和 人 A 刀 
2 一 7 十 故 站 十 2(1 一 门下 一 性 ， withr 一 4 (9.3.3) 
大 
Since D 1 一 WO0 ,一 Alf) 18 not given，we cannot get 2 directly from this 
formula (9.3.3) with 上 三 0: 


机 一 ro 十 丰 站 二 20 一 门 加 一 本 (9.3.4) 


Therefore,， we approximate the initial condition on the derivative by the central 


difference as 
双 > 一 2 


and make use of this to remove 2 from Eq. (9.3.3): 
丰 一 7 十 感人 站 十 2 一 门下 一 (一 20C0)AD) 
1 1 0 0 0 ，) 
1 一 本 CUi+1 十 2 十 (一 门 友 十 1000)Ai (9.3.0) 


We use Eq. (9.3.0) together with the initial conditions to get z and then go 
on with Eq. (9.3.3) for K = 1 2, ..… Note the following facts: 


。 We must haver < ]l to guarantee the stability. 


。 The accuracy of the solution gets better as 六 becomes larger So that Ax 
decreaseS. 


It is therefore reasonable to select 了 一 1. 
The stability condition can be obtained by Substituting Eq. (9.2.4) into 
Eq. (9.3.3) and applying the Jury test [P-3]: 


入 一 27rcos(r/P) 十 2(1 一 门 - 和 1， 和) 十 20(1 -cos(r/P)) 一 DX+1=0 
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We need the solution of this equation to be inside the unit circle for stability， 
which requires 


1 和 大 
4 一 <1 (9.3.7) 


< 一 一 一 一 个， /= < 
] 一 cos(r/P) AxX2 


The objective of the following MAILAB routine“wave()”is to Implement 
this algorithm for Solving a one-dimensional wave equation. 


卫 xample 9.4. A Hyperbolic PDE: One-Dimensional Wave (Vibration). Consider 
a one-dimensional hyperbolic PDE 


927(x， 人 四 8z2(x， 人 四 


2 9 forO0<xY<2, 0<y<2, and0<1 生 2 (E9.4.1) 
攻 


with the initial conditions and boundary conditions 


U(z,0) 一 X(1 一 X)， 0/ot(x,0) =0 fort 一 0 (上 9.4.2a) 
2U(0,1) 一 0 forx 王 0， VD =0 forx 王 1 (上 9.4.2b) 


We made the following MAILAB program“solve_wave.m” in order to use 
the routine“wave()”to Solve this equation and ran this program to get the result 
Shown in Fig. 9.3 and see a dynamic picture. 


function [u,xt] = wave(a,Xxf,T,it0,ilto,bxo,bxf,M,N) 
%Solve a U_ xx = U tt for 0<=X<=Xxf，0<=t<=T 
Initial Condition: U(x;,0) = it0(x)，UuUt(x,，0) = Ilto(x) 
Boundary Condition: u(O,t)= bxo(t)，uU(xf,t) = bxf(t) 
M=#of subintervals along Xx axis 
=#of subintervals along t axis 
X= [0:M] *dxi 
t= [0O:N]*dt; 
1，u(i,1) = ito0(x(i)); end 
1 
],k) = [bxo(t(k)); bxf(t(k))]; 


于 
肖 
1 


丰 关 站 相 二 天 和 和 人 有 六 大 全 人 TAR) 
uU(2:M,2) = ri*u(1:M- 1,1) + (1 - Pr)*u(2:M,1) + r1*u(3:M+ 1,1) ... 
+ dt*ilt0(x(2:M) ); %Eq.(9.3.6) 
fork=3:N+1 
UuU(2:M,k) = Frx*u(1:M - 1,k - 1) + r2*u(2:M,k-1) + rx*u(3:M+ 1,k - 
- U(2:M,k - 2); %5Eq.(9.3.3) 


%SO1Ve_Wave 

a= 1 

it0 = inline('x.*(1-xX)"，Xx'); itto0 = inline('0'); %(E9.4.2a) 

bxot = inline('0'); bxft = inline('0'); 5%(E9.4.2b) 

xf=1;)M=20; T=2;N=50| 

[u,xit] = wave(a,xf,T,it0,ilto,bxot,bxft,M,N) ; 

figure(1)，CcLf 

mesh(t,Xx,U) 

figure(2)，CcLf 

forn=1:Ns%dynamic picture 
plot(x,u(:,n))，axis([0 xf -0.3 0.3])，pause(0.2) 

end 
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上 上 | 
0 0 0 0.2 0.4 0.6 0.8 X 1 
(a) (b) A snap shot 


Figure 9.5 Asolution for a 1-D hyperbolic PDE obtained by using “wave ( )”(Example 9.4). 


9.3.2 Two-Dimensional Hyperbolic PDE 
In this section，we consider a two-dimensional wave equation for the amplitude 
function x(x, y,1) ((x, y) 18 posltion, ft ls time) as 
92Cc yy) 有 92 yt 9200c1 
14122257 4 2 人 92Cc 
9X2 0y2 912 
for 0<xY<XYf 0<yY<yfr 0<1 < 了 


(9.3.8) 


In order for this equation to be Solvable, we should be provided with the boundary 
conditions 


U(O, y 三 Do 加 VC yb 一 Der(y,， 
&U(X，0， 1) 一 Dyo(x，1)， and MU(X， yt 三 DrGc,h 


as well as the initial condition (xz, y, 0) 一 10(x, y) and 0U/91 0o(x, yy, 0) 一 
2(x，y). 

In the Same way as with the one-dimensional case，we Ieplace the Sec- 
ond derivatives on both Sides by their three-point central difference approxi- 
mation (9.3.1) as 


大 大 大 大 大 大 K 二 1 天 人 一 1 
页 三 有 1 一 210 十 相关 1 3 2 一 20 十 本 -1) 2 十 号 
AX2 Ay 人 12 


(3 


了 
with 大 是 三 必 记 Ay = 世 ， Af 一 二 
ZI. 人 N 


which leads to the explicit central difference method: 


2 三 六 CA 十 中 六 1) 十 2(] 一 六 一 2 十 六 CT 十 zt 1 二 t 了 1 
(9.3.10) 
和 和 A 姑 A 刀 
with 7 一 ， 全 一 
AxX2 Ay 
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Since DO 一 ULGC 记 yi 一 An) is not given, we cannot get ) directy from this 
formula (9.3.10) with K = 0: 


1 0 0 0 0 0 
2 产 (C HH1 十 只 ,六 1) 友人 由 二 和 六 牛 六 CT 牛 ID 一 1 
Therefore，we approximate the initial condition on the derivative by the central 
difference as 
人 

7 志 5 1/ 

一 一 一 一 100) ,大 9.3.12 
AT7 0( 7 y) ( ) 


and make use of this to Temove zi from Eq. (9.3.11) to have 


， 


1 1 0 0 0 0 
1 二 了 xz 十 必 广 站 十 户 Gr) 十 只- 站 } 


十 2( 一 产 一 六) 丰 ) 十 动 CD)Ai (9.3.13) 


We use this Eq. (9.3.13) together with the initial conditions to get 2 ) and then 
go on using Eq. (9.3.10) forK = 1 2,..…Asufficient condition for stability [S-1， 
Section 9.0] ls 

44A 刀 


一 一 一 -一 一 芯 9.3.14 
人 Ax2 十 A 记 一 4 ) 


The objective of the MAILAB routine“wave2()”is to Implement this algo- 
rithm for solving a two-dimensional wave equation. 


卫 xXample 9.$S. A Hyperpbolic PDE: Two-Dimensional Wave (Vibration) Over a 
Square Membrane. Consider a two-dimensional hyperbolic PDE 


1 7322(x,y， 门 了 2 yy 有 NAN 920c yy 放 
4 8x2 3y2 到 8712 
for 0<xY<2，0<y>y<2 and 0<1<2 (上 9.5.1) 


with the zero boundary conditions and the initial conditions 


2U(0 yt 一 0， UN 一 0， MU 0,1) 一 0，U21) 一 0 (上 E9.3.2) 
U(xz,y,0) 一 0.1Sin(rx)sin(ry/2)， 0/ot(xy,0) 一 0fort 王 0 (上 E9.5.3) 


We made the following MAILAB program“solve_wave2.m” in order to use 
the routine“wave2()”for solving this equation and ran this program to get the 
result Shown in Fig. 9.6 and see a dynamic picture. Note that we can be Sure of 
stability，since we have 


44A 六 4(1/4)(2/20)” 


1 
人 E9.54 
“一 XzTA 呈 一 O202 二 21207 一 2 人 
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function [u,xyit] = wave2(a,D,T,it0,ilto,bxyt,Mx,My,N) 

%Ssolve al(UXxx+Uyy) =Uttfor D(1) <=Xx<= D(2)，D(3) <=y<= D(4)，0 <= 七 <= 开 
Initial Condition: U(x,y,0) = it0(x,y)，UuUt(x,y;，0) = ilto0(x,y) 
Boundary Condition: U(x,y:t) = bxyt(x,y,t) for (xy) on Boundary 
Mx/My = # of Subintervals along x/y axis 
N=#of subintervals along t axis 

= (D(2)- D(1))/Mxij x = D(1)+[0:Mx]*dx; 
= (D(4)- D(3))/My;i yY = D(3)+[0:My] *dy; 
T/N; t = [0:N]*dt; 

s%Initialization 

U = zeros(My+1,Mx + 1); ut = zeros(MYy + 1;Mx + 1); 

for j = 2:Mx 

or 夺 = 2:My 

u(i,j) = itO(x(j),y(i)); ut(i j) = iltoO(x(j)，y(I)) 
end 


2= ardt*dtj rx = adt2/(dx*dx); ry = adt21(dy*dy); 
1- Prx - Pryi Prxy2 = FrXxy1*2; 
) 
O:N 
KAN 
Ii= 1:MYy +1s%Boundary condition 
u(i;[1 Mx+1]) = [bxyt(x(1)，y(i)t) bxyt(x(Mx + 1)，yY(i) st)]; 
end 
orj=1:Mxr+1 
u(I1 My + 1], j) = [bxyt(x(j)，y(1),t); bxyt(x(j)，y(My + 1),t)]; 
end 
ifk==0 
for = 2:My 
for j = 2:Mx %Eq.(9.3.13) 
地 证 和 和 三才 和 站 和 寺 丰 丰 - 守 站 于 下 直 未 王 本 宙 全 
+ Pry*(u1(Gi -1 j)+u1Gi+1j))) + rxyl*u(i j) + dt*ut(IT,j); 





end 
end 
el1se 
for II = 2:My 
for j = 2:MXx 55 
u(i,j) = Prx*( 
+ ry*(u_1( 


Eq.(<eqnr>9.3.10)</eqnr> 
人 
-1j)+uiG+1j)) +rxy2*u(ij) -u2(); 


U_ 
工 


end 
end 
end 
U2=uU1i u1=uUi supdate the buffer memory 
mesh(x,y,uU)，axis([0202-.1 .1])，pause 
end 


%SO1Vve_wWave2 
it0 = inline('0.1*Ssin(pi*Xx)*Sin(pi*y/2)'，X'，y'); s%(E9.5.3) 

ilt0 = inline('0'，Xx'，y'); bxyt = inline('0'，Xx'，y't')i %(E9.5.2) 
a= .25;1D= [0202];T=2;Mx= 40; MY = 40;N= 40; 

[u,x,yt] = wave2(a,xf,T,it0,ilto,bxyt,Mx,My;N) ; 


0 0 
(b) Att= 1.8 





Figure 9.6 The solution of atwo-dimensional hyperbolic PDE: vibration of asquare membrane 
(Example 9.5). 
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9.4 _ FINITE ELEMENT METHOD (FEM) FOR SOLVING PDE 


The FEM method is another procedure used in finding approximate numerical 
Solutions to BVPS/PDES. It can handle irregular boundaries in the Same way as 
regular boundaries [人 R-1, S-2, Z-1]. It consists of the following steps to Solve the 
elliptic PDE: 


827(x， 327(x， 
2 2 六 和 gc yu 习 一 Ce, y) (9.4.1) 
其 0y 


for the domain D enclosed by the boundary 有 8 on which the boundary condition 
is given as 


U(Z,yY) 一 px,y) on the boundary 妃 (9.4.2) 
1. Discretize the (two-dimensional) domain 刀 into，Say，Ny， Subregions 
{Si, 0 ...，9Ns} such as triangular elements，neither necesSsarily of the 
Same SiZze noOr necesSarily covering the entire domain completely and 
exactly. 
2. Specify the positions of N，nodes and number them starting from the 
boundary nodes, Say,) 一 1...,， No, andthen the interior nodes, Say, 7 一 
N 沁 十 1 ...，N，. 


3. Define the basis/shape/interpolation functions 


页 0 妨 三 协 交 fory 三 1 NJYGC ED (9.4.3a) 


办 sy(X， y) 二 有 LUD 十 Prs(2)X 十 Pios (3)y 
for each subregion ,yy (9.4.3b) 


collectively for all subregions 8s 王 1 : Ny and for each node 妹 二 1 : Ni, So 
that 1 1 only at node 7, and 0 at all other nodes. Then, the approxi- 
mate solution of the PDE is a linear combination of basis functions 


内 Gy7) as 


Ni Jp Nm 
(CC， 7) 王 co(r， y) 二 >》 cng Cr， y) 二 >》 cn 加 十 cng 三 cf pi 未 cy > 
二 1 一 1 1 一 No 十 1 
(9.4.4) 
where 
0 一 [和 加 gm ci=[c c cmw] (9.4.5a) 
0=[gwrl gm gw co=[cwr cwia cv 


(9.4.5b) 
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For each Subregion y 一 1 ..., As, this solution can be written as 
Ni Vn 
0 (x， y) “一 >》 cngos(， y) 二 》 ,cn(pns() 十 Dry(2)X 十 Prsy(3)y) 
1 一 1 7 一 1 
(9.4.0) 


4. Set the values of the boundary node coefficients in cl to the boundary 
values according to the boundary condition , 

35. Determine the values of the interior node coefficients in c> by Solving the 
System of equations 








42c2 一 dd (9.4.7) 
Where 
Ni 了 了 
0 0 0 0 
41 一 2 | 世 e] 区 色 站 区 色 区 双 SOxv， ea | 和 As 
(9.4.8) 
901 一 [0 gw 
0 
元 O0s 二 [plsO) Ps ， Pms (2)] 
的 
0 
页 9 一 [PisG) PsG) ， Pwps (3)] 
了 
Ni 了 交 
0 0 0 0 
42 ee 2 [we] 区 色 千 区 汉 区 汉 5 SOCxs， oo | As 
(9.4.9) 
02 一 [gwvoily gba gs 
0 
南 22s 一 [PwptlsC) bwor2s(C2) gwns(2) 攻 
0 
而 %%: = [pwo+ls(3) bwpt2s(3) gwns(G3)] 
d= -4icl 一 》 Jo yo)g2sAS (9.4.10) 


3=1 


(xs, y): the centrold (gravity center) of the sth Subreglon 


The FEM is based on the variational Principle that a solution to Eq. (9.4.1) 
can be obtained by minimlizing the functional 


8 友 8 < 
| 人 


一 SC yJ2GCc y) 十 270c 7)UCr， 中 dxdy (9.4.11) 
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which， with x(x, y) 一 cro(z, y)，can be written as 


7 0 0 
7 一 二 es 
大 二 元 07 人 人 
一 SC,y)crop7rc 十 270x， we] dx dy (9.4.12) 


The condition for this functional to be minimized with respect to c ls 


1= 有 /| 有 0 
dc 久 人 有 


一 8C, y)o20 cec 十 Cr， oo dxdy 一 0 (9.4.13) 
AN 
多 41c1 十 42c2 十 》， Js y)02vASy 一 0 (9.4.14) 


3 


See [R-1] for details. 

The objectives of the MAILLAB routines “fem_basis_ ftn()” and 
“fem_coef()”are to construct the basis function ms(x, y)7S for each node 
]. 一 1,..., Nandeach subregionsy 一 1 ..., Asandto getthe coefficient vector 
c of the solution (9.4.4) via Eq. (9.4.7) and the solution polynomial (xz，y) 7S 
Via Eq. (9.4.6) for each Subregion 8 一 1, ...， As, Tespectively. 

Before going into a Specific example of applying the FEM method to solve 
a PDE, let us take a look at the basis (Shape) function 内 (x, y) for each node 
. 一 1,...,， Ni, which is defined collectively for all of the (triangular) subreglions 
So that js 1 only at node 1, and 0 at all other nodes and can be generated by 
the routine“fem_basis_ftn()”. 


function p = fem basis _ftn(N,S) 
%5p(i,s,1:3): coefficients of each basis ftn phi 1IL 
for S-th subregion(triangle) 
: X&y coordinates of the n-th node 
: the node #s of the Ss-th subregion(triang1le) 
1); 55 the total number of nodes 
); 55 the total number of Subregions (triangles) 


区 
人 


Size(N 

size(S',1 
:N_ 
1 : 
1 : 


了 
n 
N_ 
1 
3 

%The nth basis ftn is 1 only at node n,， 


for i=1:3，pln,s,i) = pnt(Ii); end 
end 
end 
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function [U,c] = fem_coef(f,g,p,c;N,S;N_i) 
s%5p(is,1:3): coefficients of basis ftn phi ii for the s-th subregion 
scC=[.11.00.]with value for boundary and0for interior nodes 
%5Nn,1:2) : X&Yy coordinates of the n-th node 
%5S(S,1:3) : the node #Ss of the S-th Subregion(triangle) 
%N_ 工 : the number of the interior nodes 
%U(S,1:3) : the coefficients of p1 + p2(S)X + p3(S)y for each Subregion 
= Size(N,1); 55 the total number of nodes = Nb+ Ni 
Size(S,1); % the total number of subregions (triangles) 
eros(N_i,1); 
N_n-N_i 


(S(s,1)，:) +N(S(s,2)，:) + N(S(s,3)，:))/13; %5gravity center 
s%5phi_ ixx*phi_n,x + phi iyyx*phi ny - g(x,y)*phi ix*phi_n 
p_vctr = [p([inl's,1) p([i nl,s,2) p([i n],s,3)]; 
tmpg(s) = Sum(p(Iiys,2:3).*p(nys,2:3))... 
-g(xy(1) ,xy(2))*p_vCctr(1，:)*[1 xy] *p_vctr(2，:)*[1 xy] 
dS(sS) = det([N(S(S,1)，:) 1; N(S(S,2)，:) 15N(S(S,3)，:) 1])712; 
%area of triangular Subregion 
ifn== 1，tmpf(S) = -f(xy(1),xy(2))*p_vctr(1，:)*x[1 xy]' end 
end 
Al12(I - Nb,n) = tmpgxabs(dS)'; %5Eqs. (9.4.8),(9.4.9) 
end 
d(i-N_b) = tmpfx*abs(dS)' ; %5Eq.(9.4.10) 
end 


12(1:N_i,1:N_b)*xc(1:N_b)'; %5Eq.(9.4.10) 


:N_n) = A12(1:N 工 
:N_Ss 
1:3，U(sS,j) = cx*p(:,S,j); end %Eq.(9.4.6) 


N_b+1:N_n)N\d;i ssEq.(9.4.7) 


中 十 


一 





Actually, we will plot the basis (Shape) functions for the region divided into four 
triangular Subregions as depicted in Fig. 9.7 in two wayS. First， we generate the 
basis functions by using the routine“fem_basis_ftn()”and plot one of them 
for node 1 by using the MAILAB command mesh()，as depicted in Fig. 9.8a. 
Second, without generating the basis functions，we use the MAILAB command 
“trimesh()”to plot the shape functions for nodes7 风 一 2, 3, 4, and 3 as depicted 
in Figs. 9.8b-e，each of which is 1 only at the corresponding node 7 and is 
0 at all other nodes. Figure 9.8f is the graph of a linear combination of basis 


functions 
和 


UO 人 一 cg) 一 >》 cot 人 人 (9.4.15) 


有 一 1 


having the given value cn at each node 7. This can obtained by using the MAT- 
LAB command “trimesh()”as 


>>trimesh(S,N(:,1),N(:, 2)，c) 


where the first input argument S has the node numbers for each Subregion, the 
Second/third input argument N has the x/y coordinates for each node,， and the 
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=2 coordinates of 


nodes 
N= 广 1 1; 
本 1 1; 
1 一 1; 
一 1 一 1; 
0.2 0.5] 


node numbers 
of subregions 














J S=[1 2 5 
2 3 5 
3 4 5 
站 1 4 5] 
1 1 1 1 nm=3 
丑 -0.5 0 0.5 


Figure 9.7 Aregion (domain) divided into four triangular subregions. 


fourth input argument c has the function values at each node as follows: 


1 -1 1 0 
关 汪 1 1 1 
9 一 ，， N=l1l -1 1， c=|l2 | (0.416) 
3 4 5 
让 -1 -1 3 
0.2 ”0.5 0 


For this job，we make the following program“show_basis.m” and run it to 
get Figs. 9.7 and 9.8 together with the coefficients of each basis function as 


-3/10 0 0 人 
-7/10 -7/16 0 | 


P(, 1) 一 0 3/10 1/10 
0 0 7/30 72 
2 5/4 2/3 5/6 
-1l/2 0 0 
1/2 15/16 0 | 
pn)=| 0 5/16 1/2 (9.4.17) 
0 0 -1/2 -3504 | 
0 -54 0 5/6 
4/5 0 0 112 
6/5 1/2 0 0 
pC23)=| 0 -LU2 -2/5 0 


0 0 -4/15 -1/2 
32 0 2/3 0 


The meaning of this N，(the number of nodes:$) x N，(the number of subre- 
gions:4) x 3 array P is that, Say, the second rows of the three Sub-arrays constitute 
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1 
(a) 人 1 (x 吃 (b) 4g2(% 外) 








-T-1 = 人 全 1 
(c) ya(X y) (d) ga(x Y) 








一 4 -1 -1 
(e) %5(XyY) (D) go(X Y)+24s(X Y)+344(0X Y) 


Figure 9.8 The basis (shape) functions for nodes in Fig. 9.7 and a composite function. 


the _ coefficient vectors of the basis functon for node 2 as 


一 7/10 十 (L/2)x 十 (6/3)y》 for subregion 1 

一 7/16 十 (13/16)x 十 (1/2)y for Subregion > 

内 坝 二 0 十 0.X 十 0.y for Subregion 3 
0 十 0.X 十 0.y for Subregion 4 


(9.4.18) 


which turns out to be 1 only at node 2 [li.e., (1,1)] and 0O at all other nodes and on 
the Subregions that do not have node 2 as their vertex, as depicted in Fig. 9.8b. 
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With the program “show_basis,m” in your computer, type the following com- 
mands into the MAILAB command window and see the graphical/textual outpnut. 


>>Show_basis 
>>p 


Now, let us see the following example. 


%Show_basis 
ClLear 
N= [-11;11;1 -1;-1 -1;0.2 0.5]; %the list of nodes in Fig.9.7 


= Size(N,1); sthe number of nodes 
[125;235;345;145]; %the 1Listof subregions in Fig.9.7 
NSs = Size(S,1); “the number of subregions 
figure(1)，clLf 
:N_S 
[S(s,:) S(s,1)]; 
13:3 
plot([N(nodes(I),1) NCnodes(I + 1)，,1)]， 
[N(nodes(i),2) N(nodes(i+1),2)])，hold on 
end 
end 


s%basis/shape function 
p = fem basis _ftn(N,S) ; 
x0= -1;j xf=1;yo0= -1;yf= 1; %graphic region 
figure(2)，CclLf 
Mx = 50; My = 50; 
dX (xf - x0)/Mxi dy = (yf - yYO)/VMYyi 
XI x0O + [0:Mxlj*dxj yz= yo+ [0:My]*dyi 
Ins= [12345]; s%the listof node numbers whose basis ftn to plot 
for itr = 1:5 
In= ins(itr); 
if itr == 
for TIL = 1:1length(xI) 
for j] = 1:1length(yI) 
Z(j,i) = 0 
for sS= 1:N Ss 
if inpolygon(xi(i),yi(j)，NGS(s,:)，1) NGS(S,:)，2)) > 0 
2Z(j,i) = pl(in's,1) + p(inSs,2)*xi(i) + p(i ns,3)*yI(j); 
break ; 
end 
end 
end 
end 
Subplot(321)，mesh(xi,yiy,Z) %basis function for node 1 
elSe 
cl1 = zeros(Size(c)); cl1(in) = 1; 
Subplot(320 + itr) 
trimesh(S,N(:,1),N(:,2),c1) %basis function for node 2-5 
end 
end 


cC=[01230]; s%the values for all nodes 
Subplot(326) 
trimesh(S,N(:,1),N(:,2),c) %5Fig.9.8f: a composite function 
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上 Example 9.6. Laplace"s Equation: Electric Potential Over a Plate with Point 
Charge. Consider the following Laplace'`s equation: 


32 y) ， 9270c，y) 
下 0 一 帮 Cc, y) (上 E9.0.1) 


for 一 1 和 X 挟 十 1 一 1 和 yy 入 十 1 


V270r， JJ) 


where 
一 ] _ for (xz, y) = (0.3, 0.3) 
Jo y) 王 1 二 1 for (xy)=( 一 0.3, 一 0.3) (E9.6.2) 
0 elsewhere 


and the boundary condition is (xz, y) =0 for al boundaries of the rectangu- 
lar domain. 

In order to Solve this equation by using the FEM，we locate 12 boundary 
points and 19 interior points, number them, and divide the domain into 36 tri- 
angular Subregions as depicted in Fig. 9.9. Note that we have made the Size of 
the Subregions Small and their density high around the points (十 0.3, 十 0.3) and 




















-1 -0.5 0 0.5 X 1 





Figure 9.9 An example of triangular subregions for FEM. 
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(一 0.5, 一 0.3), Since they are only two points at which the value of the right-hand 
Slide of Eq. (9.6.1J) is not zero, and consequently the value of the solution (xy) 
is expected to change Sensitively around them. 

We made the following MAILAB program“do_fem.m” in order to use the 
routines“fem_basis_ftn()”and“fem_coef()”for solving this equation. For 
comparison, we have added the statements to solve the same equation by using the 
routine“poisson()”(9Section 9.1). The resujlts obtained by running this program 
are depicted in Fig. 9.10a--c. 








(c) 16x15-point FDM (Finite Difference Method) solution 


Figure 9.10 Results of Example 9.6. 
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%do_Tfem 
% for Example 9.6 
Clear 
N= [-10;-1 -1;-1/2 -1;0 -1;1/2 -1; 1 -1;1 0;1 1;1/2 1; 0 1 
-1/2 1;-1 1; -1/2 -1/4; -5/8 -7/116;-3/4 -5/8j-1/2 -5/18; 
-1/4 -5/8j-3/8 -7/116; 0 0;j 1/2 1/4;5/18 7/116;3/14 5/18; 
5/8;1/4 5/8;3/18 7/116;-9/16 -17/132j;-7/116 -17/132; 
-7/116;9/116 17/32;7/116 17/32;1/2 7/116]; %nodes 
; %the number of boundary nodes 
12;1 11 19;10 11 19;4 5 19;5 7 19;) 5 6 7;j12 15;)23 15; 
17;3 4 17;44 17 19;13 17 19;1 13 19;1 13 15;7 8 22;8 9 22; 
24;9 10 24; 10 19 24; 19 20 24;7 19 20; 7 20 22j;13 14 18; 
14 15 16;16 17 18;20 21 25;21 22 23;23 24 25j;14 26 28; 
16 26 27;18 27 28; 21 29 31;23 29 30;25 30 31; 
26 27 28; 29 30 31]; %triangular subregions 


f962 = (norm([x yl]+[0.5 0.5])<0.01)-(norm([x y]-[0.5 0.5]) < 0.01) ; 


f=inline(f962，Xx'，y'); 5%(E9.6.2) 
g=inline( 0 ，Xx' ，y' )i 
Size(N,1); %the total number of nodes 
Nn- Nb;i %the number of interior nodes 
zeros(1,N_n); %boundary value or 0 for boundary/interior nodes 
= fem_basis_ ftn(N,S) ; 
[U,c] = fem_coef(f,g,p;c;N,S,N_I); 


soOutput through the triangular mesh-type graph 
figure(1)，clLf，trimesh(S,N(: ,1),N(:，2)，Cc) 


soOutput through the rectangular mesh-type graph 
N_ s = Size(S,1); %the total number of subregions (triangles) 
x0O0= -1j xf= 1jyo= -1)yf= 1 
Mx = 16;j dx= (xf - xX0O)/MXx; xi= XO+[O0:MXx]xdx; 
My = 16; dy= (yYf -yo0)/Myi yi= yo+[0:My]*dyi 
for TIL = 1:1length(xi) 
for j = 1:1length(yI) 
forS= 1:NSss%which subregion the point belongs to 
if inpolygon(xi(i),yi(j)，N(S(s,:)，1),N(S(s,:),2)) > 0 
Z(i,j) = U(Ss,:)*[1 xi(i) yi(j)] ; %Eq.(9.4.5b) 
break; 
end 
end 
end 
end 
figure(2)，clf，mesh(xi,yIi)Z) 
#%For comparison 
bx0o = inline('0'); bxf inline(' 0 ); 
by0 = inline('0'); byf inline(' 0 ); 
[U,x,y] = poisson(f,g,bxo,bxf,by0o,byf,[xo xf yo yf],Mx,My) ; 
figure(3)，clLf，mesh(x,y)U) 
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In this section, we will see what problems can be Solved by using the GUI (graphic 
user interface) tool of MAILAB for PDEs and then apply the tool to solve the 
elliptic/parabolic/hyperbolic equations dealt with in Examples 9.1/9.3/9.5 and 9.0. 
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9.5.1 Basic PDESs Solvable by PDETOOL 
Basically, the PDE toolpbpox can be used for the following kinds of PDE. 


1. Elliptic PDE 
一 V.(cVI 由 十 aU 一 大 over adomain 2 (9.5.1) 
with Some boundary conditions like 


Au 一 r (Dirichlet condition) 
(9.3.2) 
Or nn:cV 十 gg 一 8& (generalized Neumann condition) 


on the boundary 042,， where n ls the outward unit normal vector to the boundary， 


Note that, in case w is a scalar-valued function on a rectangular domain as 
depicted in Fig. 9.1，Eq. (9.3.1) becomes 


本 (一 思 32Cc 


5 5 ) 十 ao 妨 = Je) (9.5.3) 


and if the boundary condition for the leftrside boundary segment ls of Neumann 
type like Eq. (9.1.7)，Eq. (9.3.2) can be written as 


2 区 十 guGr,y) 
0OX 07y 
0 ? 
-2 上 du y) 一 SG y) (9.5.4) 


Since the outward unit normal vector to the left-side boundary is nm 一 1 where ii 
and j are the unit vectors along the x axis and y-axis, respectively， 


2. Parabolic PDE 


0 
一 V.(cVa) 十 QU 十 4 三 : 矿 (9.3.9) 


over a domain 4 and for atime range0 生 1 上 莹 了 


with boundary conditions like Eq. (9.5.2) and, additionally, the initial condition 
2 (10). 
3. Hyperbolic PDE 


92 


一 V . (cV2 十 QU 十 4 三 < 大 (9.3.0) 


over a domain 4 and for atime range0 和 1 上 了 
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with boundary conditions like Eq. (9.3.2) and, additionally, the initial conditions 
M(to)/ (to)， 


4.Eigenmode PDE 


一 V.(cVi) 十 CU 一 和 QU (9.3.7) 


over a domain 2 and for an unknown elgenvalue 人 


with Some boundary conditions like Eq. (9.3.2). 


The PDE toolbox can also deal with a system of PDESs like 





一 V. (clIVU1) 一 V (ci2Vai2) 十 GUI 十 ai2U2 三 万 
Over a domain S2 
一 V . (ciVIU) 一 V. (cz2Vz2) 十 0G21UI 十 022l2 一 户 


(9.5.8) 





with Dirichlet boundary conditions like 


ji ji2 | | 二 广 
一 9.5.9 
的 ]22 | | z2 户 0 
or generalized Neumann boundary conditions like 


n'. (cl1VIU1) 十 n: (ci2Vzi2) 十 91IUI 十 9l272 一 81 
(9.5.10) 
n. (C21VIU1) 十 nn (cz2Vu2) 十 9g21U1 十 92272 一 82 


or Inixed boundary conditions，where 
三 Cl11 “5C12 ， 记 二 CI11 412 ， 太 = 万 ， 基 三 UL1 
C21  C22 021 022 万 12 
ji11 712 方 911 912 81 
三 3 人 过 当 > 
人 122 户 2 921 922 有 32 


9.5.2 The Usage of PDETOOL 


The PDEtool in MAILAB solves PDEs by using the FEM (fnite element method). 
We Should take the following steps to use it. 


0. Type “pdetoo1l ”into the MAILAB command window to have the PDE 
toolbox window on the Screen as depicted in Fig. 9.11. You can tog- 
gle on/off the grid by clicking “Grid in the Options pull-down menu 
(Fig. 9.12a). You can also adjust the ranges of the x axis and the y axis 
in the box window opened by clicking "Axes_Limits” in the Options pull- 
down menu. If you want the rectangles to be aligned with the grid lines， 
click “Snap(-to-grid)”in the Options pull-down menu (Fig. 9.12a). 玉 you 
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Info， Select the tppe efPOE application from this Pop-up menu， | 








Figure 9.11 The GUI (graphical user interface) window of the MATLAB PDEtool. 


want to have the x axis and the y axis of equal scale so that a circle/square 

may not look like an ellipse/rectangle, click “Axes_Equal in the Options 

pull-down menu. You can choose the type of PDE problem you want to 

Solve in the Submenu popped out by clicking “Application ”in the Options 

pull-down menu (Fig. 9.12a). 

(cf) In order to be able to Specify the boundary condition for a boundary Segment 
by clicking it, the segment must be inside in the graphic region of PDEtool. 


1. In Draw mode, you can create the two-dimensional geometry of domain 人 2 
by using the constructive solid geometry (CSG) paradigm，which enables 
us to make a Set of solid objects such as rectangles,，circles/ellipses，and 
polygons. In order to do So，click the object that you want to draw in 
the Draw pull-down menu (Fig. 9.12b) or click the button with the cor- 
responding icon ( 口 ], 田 , 心 , 竺 ;有 ) in the tool-bar just below the top 
menu-bar (Fig. 9.11). Then，you can click-and-drag to create/move the 
object of any Size at any position as you like. Once an object is drawn， 
it can be selected by clicking on lt. Note that the Selected object becomes 
SuUIrounded by a black solid line and can be deleted by pressing Delete or 
^R(Ctrl-R) key. The created object is automatically labeled, but it can be 
relabeled and resized (numerically) through the Object dialog box opened 


书 ptions 
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Figure 9.12 Pull-down menu from the top menu and its submenu of the MATLAB PDEtool. 
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by double-clicking the object and even rotated (numerically) through the 
box opened by clicking “Rotate ”in the Draw pull-down menu. After cre- 
ating and positioning the objects, you can make a C9G model by editing 
the set formula appropriately in the set formula field of the second line 
below the top menu-bar to take the union (by default)，the intersection， 
and the set difference of the objects to form the shape of the domain S2 
(Fig. 9.11). 于 you want to see the overall shape of the domain you created， 
click "Boundary mode'” in the Boundary pull-down mennu. 


. In Boundary mode，you can remove the Subdomain borders that are 


induced by the intersections of the Solid objects，but are not between 
different materials and also Specify the boundary condition for each 
boundary segment. First，cjlick the 0S button in the tool-bar (Fig. 9.11) 
or “Boundary mode(^B)” in the Boundary pull-down menu (Fig. 9.12c)， 
which will make the boundary Segments appear with red/blue/green colors 
(indicating Dirichlet(defaultb/Neumann/mixed type of boundary condition) 
and arrows toward its end (for the case where the boundary condition 
1$S parameterlized along the boundary)， When you want to Temove all 
the Subdomain borders，click “Remove_All Subdomain _ Borders” in the 
Boundary pull-down menu. You can set the parameters 有 or 8g,9 
in Eq. (9.3.2) to a constant or a function of x and y Specifying the 
boundary condition，through the box window opened by double-clicking 
each boundary Segment. In case you want to Specify/change the boundary 
condition for multiple segments at a time，you had better use Shift-click 
the Segments to select all of them (which will be colored black) and click 
again on one of them to get the boundary condition dialog box. 


. In PDE mode, you can specify the type of PDE (Elliptic/Parabolic/Hyper- 


bolic/Eigenmode) and its parameterSs. In order to do So，open the PDE 
Specification dialog box by clicking the PDE button in the tool-bar or 
了 PDE_Specification” in the PDE pull-down menu (Fig. 9.12d)，check the 
type of PDE, and set its parameters in Eq. (9.3.1)/(9.3.5)/(9.3.6)/(9.5.7). 


. In Mesh mode，you can create the triangular mesh for the domain 


drawn in Draw mode by just clicking the A button in the tool-bar or 
Initialize_ Mesh(^D” in the Mesh pull-down menu (Fig. 9.12e). To improve 
the accuracy of the solution，you can refine SuccesSslvely the mesh by 
cllicking the 总 button in the tool-bar or Reftine Mesh 人 AM)” in the Mesh 
pull-down menu. You can jiggle the mesh by clicking “Jiggle_ Mesh” im 
expectation of better accuracy. You can also undo any refinement by 
clicking“Undo_Mesh_Change'” in the Mesh pull-down menu. 


. In Solve mode, you can Solve the PDE and plot the result by just clicking 


the = button in the tool-bar or “Solve_ PDE(E)” in the Solve pull-down 
(Fig. 9.12f. But，in the case of parabolic or hyperbolic PDE,， you mnust 
click “ Parameters” in the 9olve pull-down menu (Fig. 9.12f) to set up the 
initial conditions and the time range before solving the PDE. 
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0. Im Plot mode, you can change the plot option in the Plot selection dialog 


box opened by clicking the 克 button in the tool-bar Or Parameters”in 
the Plot pull-down menu (Fig. 9.12g). mm the Plot selection dialog box 
(Fig. 9.12h), you can Set the plot type to,， Say,，ColorHeight(3-D) and Set 
the plot style to，Say，interpolated Shading and continuous (interpolated) 
height. I you want the mesh to be Shown in the Solution graph，check 
the box of Show_mesh. In case you want to plot the graph of a known 
function, change the option(S) of the Property into “user_entry ,type in the 
MAILAB expression describing the function and click the Plot button. You 
can Save the plot parameters as the current default by clicking the Done 
button. You can also change the color map in the second line from the 
bottom of the dialog box. 


(cf) We can extract the parameters involved in the domain geometry by clicking Export… 
in the Draw pull-down menu, the parameters Specifying the boundary by clicking 
了 Export.， in the Boundary pull-down menu, the parameters Specifying the PDE by 
clicking“Export. ”in the PDE pull-down menu, the parameters Specifying the mesh 
by clicking “Export.” in the Mesh pull-down menu, the parameters Telated to the 
solution by clicking “Export.” in the Solve pull-down menu，and the parameters 
related to the graph by clicking “Export. ”in the Plot pull-down menu. Whenever 
you want to Save what you have worked in PDEtool, you may Select File/Save in 
the top menu-bar. 

(cf) Visit the website “http:Wwww.mathworks.comy/access/helpdesk/help/helpdesk. 
html ”for more details. 


9.5.3 Examples of Using PDETOOL to Solve PDEs 
In this section，we will make use of PDEtool to solve Some PDE problems that 


were dealt with in the previous sections. 


Example 9.7. Laplace"s Equation: Steady-state TIemperature Distribution Over 
a Plate. Consider the Laplace's equation (Example 9.1) 


927(x， 827(0x， 
0 人 昌 定 GD 


汉 
V-“u(Oc y) 三 二 5 0 for0<x<4 0<y<4 


(上 9.7.1) 
with the following boundary conditions. 


U(0, y) 一 e"” 一 cos y， U(4,y) 一 ercos4 一 ecosy (上 E9.7.2) 


2U(xz,0) 一 coSX 一 e”， WO 4) 一 ecosx 一 ercos4 (上 E9.7.3) 
The procedure for using PDEtool to solve this problem is as follows: 


0. Type“pdetool” into the MAILAB command window to have the PDE 
toolbox window on the Screen. Then, adjust the ranges of the x-axis and 
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the y-axils to [0 53] and [0 5], respectively, in the dialog box opened by 
clicking '“Axes_Limits” in the Options pull-down menu. You can also click 
“Axes_Equal ”in the Options pull-down menu to have the x axis and 
the y axis of equal scale So that a circle/square may not look like an 
ellijpse/rectangle. 


. Click the 口 button in the tool-bar and click-and-drag on the graphic 


Iegion to create a rectangle of domain. Then，in the Object dialog box 
opened by double-clicking the rectangle, set the LefVBottom/YWidthM/Height 
to 0O/0/4/4. In this case，you don't have to construct a CSG model by 
editing the set formula，because the domain consists of a Single object: 
a rectangle. 


.Click the 毁 J button in the tool-bar and double-click each boundary seg- 


ment to Specify the boundary condition as Eqs. (E9.7.2,3) in the boundary 
condition dialog box (See Fig. 9.13a). 


. Open the PDE Specification dialog box by clicking the PDE button in the 


tool-bar，check the box on the left of Elliptic as the type of PDE, and set 
its parameters in Eq. (E9.7.1) as depicted in Fig. 9.13b. 


. Click the A button in the tool-bar to divide the domain into a number of 


triangular Subdomains to get the triangular mesh as depicted in Fig. 9.13c. 
You can click the 闻 button in the tool-bar to refine the mesh SUcceSSivVely 
for better accuracy. 


.Click the = button in the tool-bar to plot the solution in the form of two- 


dimensional graph with the value of x(x, y) Shown in color. 


. 开 you want to plot the solution in the form of a three-dimensional graph 


with the value of x(x, y) Shown in height as well as color,，check the box 
before Height on the far-left side of the Plot selection dialog box opened 


by clicking the 史 button in the tool-bar. If you want the mesh shown in 
the Solution plot as Fig. 9.13d，check the box before Show_mesh on the 
far-left and low side and click the Plot button at the bottom of the Plot 
Selection dialog box (Fig. 9.12h). You can compare the result with that of 
Example 9.3 depicted in Fig. 9.4. 


. 开 you have the true analytical solution 


WU, y) 一 ecoSX 一 er cosy (上 9.7.4) 


and you want to plot the difference between the PDEtool (FEEMI) solution 
and the true analytical solution, change the entry “ur into “user entry” in the 
ColorContour row and the Height row of the Property column and write 
“u-(exp(y).*cos(x)-exp(x).*cos(y))”into the corresponding fields in 
the User_entry column of the Plot selection dialog box opened by clicking 


the 壤 button in the tool-bar and click the Plot button at the bottom of the 
dialog box. 
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(b) PDE specification 





(d) The mesh plot of solution 


Figure 9.13 ”Procedure and results of using PDEtool for Example 9.1/9.7. 
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卫 xample 9.8. A Parabolic PDE: Two-Dimensional TIemperature Diffusion Over 
a Plate. Consider a two-dimensional parabolic PDE 


10-4 927(x， y，, 71) 兴 927(x， yy 四) 下 OU(xC，y， 1 
0x2 97y2 01 
for 0<x<4，0<y>y<4 儿 0<1<3000 (上 9.8.1) 


with the initial conditions and boundary conditions 


WU y,0) 一 0 fort 一 0 (E9.8.2a) 
U(z, yi 一 e"coSsXY 一 ercogy forx 王 0, xx=4 >》=0, > 王 4 (E9.8.2b) 


The procedure for using the PDEtool to solve this problem is as follows. 


0-2. Do exactly the Same things as steps 0-2 for the case of an elliptic PDE 
in Example 9.7. 

3. Open the PDE specification dialog box by clicking the PDE button， 
check the box on the left of "Parabolic” as the type of PDE and set its 
parameters in Eq. (E9.8.1) as depicted in Fig. 9.14a. 

4. Exactly as in step 4 (for the case of elliptic PDE) in Example 9.7, click 
the A button to get the triangular mesh. You can click the 总 button to 
refine the mesh SuccesSively for better accuracy. 

35. Unlike the case of an elliptic PDE，you must click “Parameters”in 
the Solve pull-down menu (Fig. 9.12f) to Set the time range，Say，a3aS 
0:100:5000 and the initial conditions as Eq. (E9.8.2a) before clicking 
the = button to solve the PDE. (See Fig. 9.14b.) 


0. As in step 6 of Example 9.7, you can check the box before Height in 


the Plot selection dialog box opened by clicking the 确 button，check 
the box before Show_mesh, and click the Plot button. I you want to 
plot the solution graph at a time other than the final tme, select the time 
for plot from 

{0, 100, 200, ... ,500} 


in the far-right field of the Plot selection dialog box and click the Plot 
button again. 玉 you want to see a movie-like dynamic Picture of the 
Solution graph, check the box before Animation, click Options right after 
Animation, fll in the fields of animation rate in fps (le., the number of 
frames per Second and the number of repeats in the Animation Options 
dialog box), click the OK button, and then click the Plot button in the 
Plot selection dialog box. 


(cf) If the dynamic picture is too oblong, you can Scale up/down the solution by chang- 
ing the Property of the Height row from ”uinto “user entry” and flling in the 
Corresponding field of User_entry with, Say，"u/23"” in the Plot selection dialog box. 


4 PDE Specification 
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Figure 9.14 Procedure and results of using PDEtool for Example 9.3/9.8. 
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According to your Selection，you will see a movie-like dynamic Picture Or 
the (final) solution graph like Fig. 9.14d，which is the Steady-state Solution 
for Eq. (E9.8.1) with 09x(x, yy, 1)/01 = 0, virtually the Same as the elliptic PDE 
(E9.7.1) whose solution ls depicted in Fig. 9.13d. 

Before closing this example, let us have an experience of exporting the values 
of some parameters. For example，we extract the mesh data {p, e, tj by clicking 
“Export_ Mesh ”in the Mesh pull-down menu and then clicking the OK button 
in the Export dialog box. Among the mesh data, the matrix pP contains the X 
and yy coordinates in the first and second rowS，respectively. We also extract 
the solution u by clicking“Export_ Solution ” in the Solve pull-down menu and 
then clicking the OK button in the Export dialog box. Now，we can estimate 
how far the graphicalnumerical solution deviates from the true steady-state Solu- 
tion WU(x,y) 一 ecosXY 一 ecogy by typing the following statements into the 
MAILAB command window， 


p(2.: )'; %5x.y coordinates of nodes in column Vector 
(X) - exp(Xx).*cos(y) - U(:.end); %5dqeviation from true So1 
(err)) %maximum absolute error 


>>err = exp(y) ,.*C 


>>X = p(1.:)';)y = 
*xCOS 
>>err_max = max(abs 


Note that the dimension of the solution matrix w is 177 x 9$1 and the solution 
at the final stage ls stored in its last column u(: ,end)，where 177 is the number 
of nodes in the triangular mesh and 51 = 9000/100 十 1 is the number of frames 
Or time StagesS. 


了 kxample 9.9. A Hyperpbolic PDE: Two-Dimensional Wave (Vibration) Over a 
Square Membrane. Consider a two-dimensional hyperbolic PDE 


1 7a22Cc, yy 人 汪 924(c，y， 八 9z2(c, y， 八 
4 8x2 3y2 加 9812 
for 0<x< 2 0<y<2， and0<! 上 <2 (上 9.9.1) 


with the zero boundary conditions and the initial conditions 
U(0O,y,1) 一 0， VCO2yh) 王 0，U 01 人 一 0，UO2,1) 一 0 (上 E9.9.2) 
2U(x,y,0) 一 0.1sin(rx)Sinry/2)， 0U/ot(xy,0) =0 for 一 0(E9.9.3) 
The procedure for using the PDEtool to solve this problem is as follows: 


0-2. Dothesamethings as steps0-2forthecaseofellipticPDEinExample 9.7， 
except for the following. 

。 9et the ranges of the x axis-and the y-axis to [0 3] and [0 3]. 

。 et the LefVBottom/WidthMHeight to Q/O/2/2 in the Object dialog box 
opened by double-clicking the rectangle. 

。 et the boundary condition to zero as Specified by Eqs. (E9.9.2) in 
the boundary condition dialog box opened by clicking the 94 button 
in the tool-bar, shiftrclicking the four boundary segments and double- 
cllicking one of the boundary segments. 
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(c) The box window popped out by clicking Parameters in the Plot pull-down menu 





(d1) The mesh plot of soution 1= 0.1 (d2) The mesh plot of solution at = 1.7 
Figure 9.15 ”Procedure and results of using PDEtool for Example 9.5/9.9. 


3. Open the PDE Specification dialog box by clicking the PDE button， 
check the box on the left of `Hyperbolic” as the type of PDE, and set 
its parameters in Eq. (上 E9.9.1) as depicted in Fig. 9.13a. 

4. Do the same thing as Step 4 for the case of elliptic PDE in Example 9.8. 

5$. Similarly to the case of a parabolic PDE, you mnust click “Parameters” 
in the Solve pull-down menu (Fig. 9.12f) to set the time Tange，Say，as 
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0:0.1:2 and the initial conditions as Eq. (E9.9.3) before clicking the 三 
button to solve the PDE. (See Fig. 9.15b.) 


6060. Do almost the Same thing as Step 6 for the case of parapbolic PDE in 
Example 9.8. 


Finally, you could see the Solution graphs like Figs. 9.13(dl)&(d2), that are 
Similar to Figs. 9.0(a)(c). 


了 kxample 9.10. Laplace'"s Equation: Electric Potential Over a Plate with Point 
Charge. Consider the Laplace's equation (dealt with in Example 9.0) 


82 y) 3920 y) 
人 Je y) 


for -1<x<+l-1<y<+l 4E9.10.1) 


V27 (xc， y) 一 


Where 
一 1] _ for (xz, y) = (0.3, 0.3) 
xc 放 =1+L for (cy)=( 一 0.3, 一 0.5) (E9.10.2) 
0 elsewhere 


and the boundary condition is (xz, y) =0 for all boundaries of the rectangu- 
lar domain. 
The procedure for using the PDEtool to solve this problem is as follows. 


0-2. Do the same thing as step 0-2 for the case of elliptic PDE in Example 9.7， 
except for the following. 

。 et the LefVyBottom/Width/Height to 一 1]/ 一 LU2/2 in the Object dialog 
box opened by double-clicking the rectangle. 

。 et the boundary condition to zero in the boundary condition dialog box 
opened by cjlicking the 94 button in the tool-bar, Shift-clicking the four 
boundary segments, and double-clicking one of the boundary segments. 

3. Open the PDE specification dialog box by clicking the PDE button, check 
the box on the left of "Elliptic” as the type of PDE, and set lts parameters 
in Eq. (E9.10.1,2) as depicted in Fig. 9.10a. 

4. Click the A button to initialize the triangular mesh. 

3. Click the 史 button to open the Plot selection dialog box, check the box 
before "Height , and check the box before "Show_mesh in the dialog box. 

0. Click the Plot button to get the solution graph as depicted in Fig. 9.16c. 


7. Click Parameters in the Solve pull-down menu to open the “Solve Param- 
eterS”dialog box depicted in Fig. 9.16b，check the box on the left of 
“Adaptive mode , and click the OK button in order to activate the adaptive 
mesh mode. 


8. Click the = button to get a solution graph with the adaptive mesh. 
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Figure 9.16 ”Procedure and results of using PDEtool for Example 9.6/9.10. 


443 


444 PARTIAL DIFFERENTIAL EQUATIONS 


9. Noting that the Solution is not the right one for the point charge distribution 
given by (上 E9.10.2), reopen the PDE Specification dialog box by clicking 
the PDE button and rewrite f as below. 


f |(-((x + 0.5)"2 + (y+0.5)*2 < 0.00064) + (((x - 0.5) “2 + {y - 0.5)"2) < 0.00064)) 














10. Noting that the mesh has already been refined in the adaptive way to 
yield smaller meshes in the reglion where the Slope of the solution is 
Steeper, click Parameters in the Solve pull-down menu to open the “Solve 
Parameters” dialog box, uncheck the box on the left of "Adaptive mode ， 
and click the OK button in the dialog box in order to inactivate the 
adaptive mesh mode. 

11. Click the = button to get the solution graph as depicted in Fig. 9.106d. 

12. You can click “Reftine Mesh(M)” in the Mesh pull-down menu and click 
the = button to get a more refined solution graph (with higher resolution) 
as many times as you want， 


PROBLEMS 


9.1 Elliptic PDESs: Poisson Equations 
Use the routine“poisson()”(in Section 9.1) to Solve the following PDEs 
and plot the Solutions by using the MAILAB command “mesh ()”. 

32x(0c, y) 肾 92x(0c, y) 加 

07X2 ay2 


forO0<xY<10<y><1 


(a) VC, y) = x 十》 (P9.1.1) 


with the boundary conditions 


2U(0,) 一 六 ， ay)=1|， 
U(t, 0) 一 x2， WU, ID 一 1 (P9.1.2) 


Divide the solution region (domain) into WM; x My =3 x 10 sections. 


927(0x， y) 827(0x， y) 


了 5 一 12.5 关 32(x， y) 


(b) 


5 9 
一 一 2Sr2 cos (到 COS ( 王 )) for 0<x,y<0.4 (P9.1.3) 
with the boundary conditions 


U(0, y) 一 coOS (三 )) 2U(0.4, y) 一 一 coS (三 )) (P9.1.4) 


2U(XY, 0) 一 coOS (到 ") 2U(Xz, 0.4) 一 一 coOS (三 (P9.1.5) 
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Divide the solution region into Ms x AM = 40 x 40 sections. 
32x0c,y) 920,y) 
0x2 0y 
一 4rcos(rG +T7)) for0<x<10<y<1 (Po9.1.0) 


(9) 十 4r(Ct2 十 yue， y 


with the boundary conditions 


2(0, y) = sin(ry)， zy) = sin(r( 关 十 D)  (P9.1.7) 
WOc, 0) 一 Sin(rx2)， zu, 1 =sin(0rG 十 D)  (P9.1.8) 


Divide the solution region into M: x My = 40 X 40 sections. 


927(x， 927(x， 
《 浊 寺 《 妇 _ 


d 
四 DX2 0y” 


10e2x+y for0<xY<10<y><2 (P9.1.9) 


with the boundary conditions 


WU(0, y) = 2e” ， U(,， y) 一 2e2+?， 
MG 0) 一 2e2， uCt2) = 2e2 (P9.1.10) 


Divide the solution region into Ms x My = 20 x 40 sections. 


ay) 9) 


with the boundary conditions 


U(0, y) = 4cos(3y)， U(l, y) = 4e-3 cos(3y)， 
U(, 0) 一 4e 一 3 ， U(z,T/2) 一 0 (P9.1.12) 


Divide the solution region into M x My = 20 X 20 Sections. 
9.2 More General PDE Having Nonunity Coefficients 
Consider the following PDE having nonunity coefficients. 


32G， y) 太 9 Cr， y) 东 CDUC， y) 


82x7” 0x9y 37y2 十 8GC,y)UCC 7y) 三 Jx，y) 


(P9.2.1) 


Modify the routine“poisson()”so that it can Solve this kind of PDESs and 
declare it as 


function [u,x,y] = poisson_abc(ABC,f,g,bxo,bxf,by0,...，Mx,My,tol,imax ) 
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where the first input argument ABC is Supposed to carry the vector containing 
three coefficients 4, B, and C. Use the routine to solve the following PDEs 
and plot the Solutions by using the MAILAB command “mesh()”. 


927(xz，y) D 9 uC 站) 


全 5 10 for 0sx<s10<s7<1 (9292) 


with the boundary conditions 


U(0, 7 一 六 ad)=( 十 2)?， 


wx, 0) 一 4x2 二 (P9.2.3) 
0 ,y) 一 (2x 十 ]) 


Divide the solution region (domain) into M: x My = 20 x 40 sections. 


才 2 2 
m OU(0c y) 3 U(xz，y) 7 &U(x，y) 


一 0 for0<x< 和 10<y<1 


Dx2 0x0y 90y” 
(P9.2.4) 
with the boundary conditions 
U(0,) 一 e7 十 cogsy， ud =e +cosy 一 2) (P9.2.5) 


U(, 0) 一 e 开 十 cos(-2x)， xc = el 十 cos(l 一 2x) (P9.2.6) 
Divide the solution region into Ms x My = 40 x 40 sections. 


9027(xz，y) 027(xz，y) 027(xz，y) 
Dx2 0x0y 90y” 


for 0<XY 和 2,0 芝 》 苹 区 


(Cc) 


一 XYSiny 
(P9.2.7) 
with the boundary conditions 


U(0, y) = (3/4) cosy， MU(C2,y) = 一 Sin(y) 十 (3/4) cosy (P9.2.8) 
WU , 0) 一 3/4， WU 元 ) 一 一 3/4 (P9.2.9) 


Divide the solution region into M x My = 20 X 40 sections. 


8270x， y) 927(x， y) 927(x， y) 


必 =0 for0<x<10<y<1 
(d) 2 和 5 or 0 <X 芭 < < 
(P9.2.10) 
with the boundary conditions 
MU(0,y) = ye7， ay = (1+y)er2， 
(P9.2.11) 


U(t,0) 一 Xer， WU D 三 (十 Der 


Divide the solution region into M x M = 40 x 40 sections. 
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function [u,x,y] = poisson_Neuman(f,g,bx0o,bxf,by0o,byf,x0,xf,y0,yf,...) 
Neum = zeros(1,4); %Not Neumann，but Dirichlet condition by default 

if Length(x0) ，Neum(1) = ; end 
if Length(xf) ，Neum(2) ; end 
if length(y0) Neum(3) = ; end 
if length(yf) Neum(4) ; end 


dx 2 = dxx*dxij dy 2 = dy*dyj dxy2=2*(dx 2 + dy 2); 

rx = dx 2/dxy2;j ry = dy 2/dxy2; Prxy = Frxxdy 2; Prx = Prxi 
dx2 = dxx*2j dy2 = dy*2;j rydx = Fryx*dx2j rxdy = Fxxdy2; 
uU(1:My1,1:Mx1) = zeros(My1,Mx1) ; 


Sum_ of bv = 0j num = 0 
if Neum(1) == 0 %Dirichlet boundary condition 
form= 1:My1，u(m,1) = bx0o(y(m)); end #%5side a 
else s%5Neumann boundary condition 
form= 1:My1，duxa(m) = bxoly(m)); end %5dqu/dx(xoy) 
end 
if Neum(2) == 0 %Dirichlet boundary condition 


end 
if Neum(3) == 0 %Dirichlet boundary condition 
n1=1; nMI = Mx1i 
if Neum(1) == 0，u(1,1)=(u(1,1) + by0(x(1)))/2; n1 = 2; end 
if Neum(2) == 0，uU(1,Mx1)=(u(1,Mx1) + by0(x(Mx1)))/2; nM1 = Mxji end 
forn= nl:nM，u(1，n) = byo(x(n)); end %5side C 
else s#%5Neumann boundary condition 
forn= 1:Mx1，duyc(n) = byo(x(n)); end %5dqu/dy(x,y0) 
end 
if Neum(4) == 0 %Dirichlet boundary condition 


end 
for itr = 1:imax 
if Neum(1) s#sNeumann boundary condition 
for TI = 2:My 
u(i,1) = 2x*ryx*u(I,2) + Frxx(uli + 1,1) + UuU(i-1,1)) 
+ PrFxyx(G(i,1)*u(i,1) - F(i,1)) - Frydxx*duxa(i); %5(9.1.9) 
end 
if Neum(3)，u(1,1) = 2*(ry*u(1,2) +Fxxu(2,1)) 
+ Fxyx(G(1,1)*u(1,1) - F(1,1)) - Frydxx*duxal(1 
end 
if Neum(4)，uU(My1,1) = 2*x(Pyxu(My1,2) +rxx*u(My ,1)) .. 
+ PFxy*x(G(My1,1)*u(My1,1)- F(My1,1))+rxdyxduyd(1)- Frydxxduxa(My1) ; 
end 
end 
if Neum(2) s#sNeumann boundary condition 


) - Prxdy*duyc(1)3%5(9.1.11) 


end 
if Neum(3) s%sNeumann boundary condition 
for j = 2:MX 
uU(1,j) = 2*rxx*u(2，j)+ry*(u(1,j+l) + uU(1j-1)) 
+rxy*(G(1,j)*u(1j) - F(1,j)) - rxdy*duyc(j); s%(9.1.10) 
end 
end 
if Neum(4) s%sNeumann boundary condition 


end 


end 
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9.3 
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Elliptic PDEs with Neumann Boundary Condition 
Consider the PDE (E9.1.1) (dealt with in Example 9.1) 


ay) ga 


了 了 0 for0O0<xY<4,0 和 二》 莹 4 (P9.3.1) 
万 了 


with different boundary conditions of Neumann type，whlich was discussed 
in Section 9.1. Modify the routine“poisson()”so that it can deal with the 
Neumann boundary condition and declare it as 


function [u,x,y] = poisson_Neuman(f,g,bx0,bxf,by0,byf,x0,xf,y0,yf,，...) 


where the third/fourth/fhifth/sixth input arguments are Supposed to carry the 
functions of 


2U(X0, y)VUGCXCF， yy) 7y0)MAUCC yy 
OF 


OU(CC,，y)/DX|z=xro/OU(CC，y)/OX|z=rrVOUCC，y)/9y|>=yo/OU(CC, y)/O7y|y=y 


and the seventh/eighth/ninth/tenth input arguments are to carTry X0/XF/yo/yr 
or [xo 1]J/[xr HI/[yo ]J/[yr J depending on whether each boundary condition 
is of Dirichlet or Neumann type.， Use it to solve the PDE with the 
following boundary conditions and plot the Solutions by using the MATILAB 
command “mesh()”. Divide the solution region (domain) into M: x AM = 
20 X 20 Sections， 


(cf) You may refer to the related part of the program in the previous page. 


(a) ax,y)/axl,-0 二 一 cogsy， xd 一 ercos4 一 ecosy  (P9.3.2) 
80, y)/ayly-o = cosx， MG;4) 一 ecosxr 一 ercos4 (P9.3.3) 


(b) xz(0, 让 三 e7 一 cogsy， ax,y)/axl4 = -esin4 一 edcosy(P9.3.4) 
U(x, 0) 一 coSY 一 后 ，9U(x,y)/3y|y=4 一 ecosxY 十 ersin4 (P9.3.5) 
(@Jj 3 y)/axl,-o0 一 一 cogsy， MG4,)=ercos4 一 ecosy  (P9.3.6) 


WU,0) 一 cosX 一 6 0x(x， y)/ay|、， 一 e4cosx 十 ersin4 (P9.3.7) 


(d) zx(0,y) 三 e” 一 cogsy，0x(Ox， y)/ax| _， 一 一 e7sin4 一 e4 cos y (P9.3.8) 


DOU(X， 7)/ay|、。 一 COSX， WU 4) 一 edcosx 一 ercos4 (P9.3.9) 
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(e) 94(X， y)/3x| 一 一 COSy， 0U(XY， y)/3ax|，， 一 一 csin4 一 ef CoOSy 


(P9.3.10) 
OU(X， 7)/ay|、。 一 COSX， U(t,4) 一 etcosx 一 ercos4 (P9.3.11) 


(人 3xCc,y)/3xz|_ ,= 一 cosy， U(4, y) = ey cos4 一 ecosy(P9.3.12) 


OU(X， y)/ay|，。 一 COSXY， 0U(XY， 7)/ay| 4 一 e+cosx 十 ersin4 
(P9.3.13) 


(g) x(0,y7)=e7 一 cosy,3uCc,y)/axrl ;= 一 e7sin4 一 e cosy(P9.3.14) 


OU(X， 7)/ay|、。 一 COSXY， 0U(x， 7)/ay|、， 一 ecosx 十 ersin4 
(P9.3.15) 


(h) 0x(x， y)/3ax|，，。 一 一 COSy， 0(x， y)/3ax|，_， 一 一 c?sin4 一 ef COS y 
(P9.3.10) 


OU(X， 7)/ay| -0 一 COSXY， 0U(x， 7)/ay| 4 一 e4cosx 十 ersin4 
(P9.3.17) 


9.4 Parabolic PDEs: Heat Equations 


Modify the program“solve_heat,m” (in Section 9.2.3) So that it can Solve 
the following PDEs by using the explicit forward Euler method, the implicit 
backward Euler method, and the Crank-Nicholson method. 


uc 有 9u(x, 放 


一 于 


for0<x< 和 10<1<0.1 (P9.4.1) 


with the initial/boundary conditions 
WU 0) 一 X4， WU(0,71) = 0， U(, 力 一 1 (P9.4.2) 


(iD With the solution region divided into M x N = 10 xx 20 sections， 
does the explicit forward Euler method converge? What is the value 
of 一 4AI/(Az)2? 

(ii) If you increase M and N to make M xN=20Xx40 for better 
accuracy，does the explicit forward Euler method Stl converge?7 
What is the value of = 4A1/(Axz)2? 

(iii) What is the number NM of subintervals along the 1 axis that we Should 
choose in order to keep the Same value of r for M = 207 With that 
value of r, does the explicit forward Euler method converge? 
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(b) 10-5 





927(x，D 人 四 加 OU(z 四 


for0<x<10<L<60000 (P9.4.3) 
DX2 9 


with the initial/boundary conditions 


Gi 


ea 


二 


Gi 


(cC) 2 


927(xc，D 门 0 让) 


U(x,0) 三 2x 十 Sin(O2rrx)， 2(0, 四 =0， 2 hDD 三 2 (P9.4.4) 


With the solution region divided into M x N = 20 x 40 sections， 
does the explicit forward Euler method conversge? What ls the value 
of 了 一 4AI/(Ax)?2? Does the numerical stability condition (9.2.6) 
Seem to be So demanding?” 

HI you increase M and N to make M xWN=40Xx160 for better 
accuracy，does the explicit forward Euler method still converge? 
What is the value of r = 4A1/(Ax)22? Does the numerical stability 
condition (9.2.6) seem to be So demanding? 

With the solution region divided into M x N = 40 x 200 sections， 
does the explicit forward Euler method converge? What ls the value 
of 一 4AL/(Az)22 


for0<xY<T,0<1L<0.2 (P9.4.93) 
9X2 9 


with the initial/boundary conditions 


名 


二 


Gi 


2U(XZ,0) 一 Sin(27)， 2U(0,1) 一 0， 2U(Tr,1) 一 0 (P9.4.0) 


By substituting 
WU 1 一 Sin(2x)e-& (P9.4.7) 


into the above equation (P9.4.3)，verify that this is a_ Solution to 
the PDE. 

With the solution region divided into M x N = 40 x 100 sections， 
does the explicit forward Euler method converge? What is the value 
of 了 一 4AL(CAz)2? 

I you increase N (the number of Subintervals along the 1-axis) to 
125 for Improving the numerical stabjlity, does the explicit forward 
Euler method converge? What is the value ofr = 4A1/(Az)2? Use 
the MAILAB statements in the following box to find the maximum 
absolute errors of the numerical solutions obtained by the three 
methods. Which method yields the smallest error? 


inline('Sin(2*Xx)*exp(-8*t) ，Xx'，t'); s%true analytical Solution 
UO(xit) 
err = max(max(abs(ul - Uo))) 
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(iv) Ifyou increase N to 200, what is the value ofyr = 4AI/(Axz)2? Find 
the maximum absolute errors of the numerical solutions obtained 
by the three methods as in (iiil. Which method yields the small- 
est erTrOT? 


827(x 六 本 OU(X ,1) 


人 0xX7 01 





forO0<x<10<f<0.1 (P9.4.8) 


with the initial/boundary conditions 
2U(xz,0) = 一 Sin(rx) 十 Sin(3rxz)， 2101 三 0， UL ID) 三 0 (P9.4.9) 
(iD By Substituting 
2UOc 作 一 sin(rxz)Je-z7 十 Sin(37rx)e-Gm (P9.4.10) 


into Eq. (P9.4.3), verify that this is a solution to the PDE. 

(ii) With the solution region divided into M x N = 23 xX80 sections， 
does the explicit forward Euler method converge? What ls the value 
of 一 4AI/(Az)2? 

(iii) If you increase N (the number of subintervals along the 1 axls) to 
100 for improving the numerical stabjlity, does the explicit forward 
Euler method converge2? What is the value ofr = 4A1/(Axz)2? Find 
the maximum absolute errors of the numerical solutions obtained by 
the three methods as in (c)(ii)， 

(iv) Ifyou increase N to 200, what is the value ofy = 4AI/(Axz)2? Find 
the maximum absolute errors of the numerical solutions obtained by 
the three methods as in (c)(iii). Which one gained the accuracy the 
most of the three methods through increasing NM? 

9.$ Parabolic PDEs with Neumann Boundary Conditions 
Let us modify the routines“heat_exp()”“heat_imp()” and“heat_cn()” 
(in Section 9.2) So that they can accommodate the heat equation (9.2.1) with 
Neumann boundary conditions 


3uG, D/axr| = Pro(D)， GD/axr| =DorO (P9.5.1) 


(a) Consider the explicit forward Euler algorithm described by Eq. (9.2.3) 


x+1 一 7( 十 枚 由 十 (一 27) 政 


， . Ai 
for 三 12,..., AM 一 1 withy 三 4 一 一 


(P9.5.2) 
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(b 


S 必 


(C) 


PARTIAL DIFFERENTIAL EQUATIONS 


In the case of Dirichlet boundary condition，we don't need to get DT 


and DA ， because they are already given. But, in the case of the Neu- 
mann boundary condition，we must get them by using this equation for 


1 一 0and AM as 
zk 一 (tk 平 只 ) 平 伍 三 27) 区 (P9.5.3a) 





z 人 7 二 rr 二 U07_1) 十 人 一 27)uw 人 


and the boundary conditions approximated as 
大 大 
绚 | 


一 710 / 
二 一 态 ( 昌 ， 2 一 夏 一 200(OAx (P9.5.4a) 


1 八 一 7 
-和 一 ph1 (中 )， 0 = 十 20y(OAX (P9.5.4b) 
攻 


Substituting Eqs. (P9.5.4a,b) into Eq. (P98.3.3) yields 
一 27( 枚 一 芒 (DAz) 十 (1 一 27 夏 (P9.5.5a) 
MU 一 2704 1 十 br(OAz) 十 (1 一 27r)z  (P9.5.5b) 





Modify the routine“heat_exp()”so that it can use this Scheme to deal 
with the Neumann boundary conditions for solving the heat equation and 
declare it as 


function [u,xit] = heat_exp_Neuman(a,xfn;,T,iIito,bxo,bxf,M;,N) 


where the Second input argument xfn and the fifth and Sixth input argu- 
Iments bx0,bxf are SuUpposed to carry [xf 01] and Pu(1)， Dr， (1), reSpec- 
tively,iftheboundary condition at xo/xr is ofDirichleUNeumann type and 
they are also Supposed to carry [xf 1 1] and Dx (ID)， Do (1), respectively， 
这 both of the boundary conditions at xo/xyr are of Neumann type. 


Considerthe implicit backward Euler algorithm described by Eq. (9.2.13)， 
which deals with the Neumann boundary condition at the one end for 
Solving the heat equation (9.2.1). With reference to Eq. (9.2.13), modify 
the routine“heat_imp()”So that it can Solve the heat equation with the 
Neumann boundary conditions at two end points xzo and xyr and declare 
lt aS 

function [u,xit] = heat_imp_Neuman(a,xfn;,T,iIito,bxo,bxf,M;,N) 
Considerthe Crank-Nicholson algorithm described by Eq. (9.2.17), which 
deals with the Neumann boundary condition at the one end for solving the 
heat equation (9.2.1). With reference to Eq. (9.2.17), modify the routine 


“heat_cn()”so that it can Solve the heat equation with the Neumann 
boundary conditions at two end points xo and xy and declare it as 


function [u,xit] = heat_cn_Neuman(a,Xxfn;,T,ito,bxo,bxf,M;N) 
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(dJ) Solve the following heat equation with three different boundary condi- 
tions by using the three modified routines in (a), (b), (c) with M = 20;N 
= 100 and find the maximum absolute errors of the three solutions as in 
Problem 9.4(c)(Gi). 


827(0c， 门 攻 OCT 


for0<x<10<L<0.1 (P9.5.0) 
0X2? Di 


with the initial/boundary conditions 


(人 VC ,0) 一 Sin(rx)，09(x， /3x| _， 一 元 e71， 1(X， 四 | 一 0 


(9.5.7) 
的 xc,0) = sin(rz)，xUG1|， =0，3uCc,D/3xz| | = 一 Te 
(P9.5.8) 
( 介 ) zz,0) = sin(rxz)，3xGCc, DA3xz| 一元 e 
au(x,D/axz| 一 一 区 er (P9.5.9) 
Note that the true analytical Solution ls 
Oo 人 一 sin(rx)e-z 7 (P9.5.10) 


9.6 Hyperbolic PDES: Wave Equations 


Modify the program“solve_wave.m” (in Section 9.3) So that it can Solve 
the following PDEs by using the explicit forward Euler method, the implicit 
backward Euler method, and the Crank-Nicholson method. 

827(x 六 本 927(0x， 人 四 


4 
人 ) 0X? 912 


for0<x< 二 10<1f<1 (P9.0.1) 
with the initial/boundary conditions 


CO0) 王 0， uc,D/all =5Ssin(rx)， 
4(0,DD==0， adD=0 (P9.6.2) 


Note that the true analytical solution ls 
2 . 
2U(,1) 三 一 Sin(TxX)Sin(2 六 加 (P9.0.3) 
元 


(iD With the solution region divided into M x N =20Xx30 sections， 
what is the value of 了 一 4(Afn)2/(Axz)2? Use the MATLAB state- 
ments in Problem 9.4(c)(iii) to fnd the maximum absolute error of 
the Solution obtained by using the routine“wave ()”. 
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( 廊 With the solution reglion divided into M x N = 40 x 100 sections， 
what ls the value of r?7 Find the maximum absolute error of the 
numerical solution. 

(iii) If we increase M (the number of subintervals along the x axis) to 
50 for better accuracy，what is the value of 2” Find the maximum 
absolute error of the numerical solution and determine whether it 
has been Improved. 

(iv) If we increase the number AM to 32，what is the value of r 7 Can 
WwWe expect better accuracy in the light of the numerical stability 
condition (9.3.7)2 Find the maximum absolute error of the numerical 
solution and determine whether it has been Improved or not， 

(V) What do you think the best value of r is? 


927(0xz 六 927(0c，D 人 四 
ax2 902 


with the initial/boundary conditions 


(b) 6.25 for0<xY<T 0<1L<0.4r  (P9.6.4) 


xc;,0) = 一 sin(2xz)， 9xCc,D/all = 0， 
x(0,D=0， unD=0 (P9.6.5) 


Note that the true analytical Solution is 


2U(xz,1t) 三 Sin(2x)cos(91) (P9.0.0) 

(iD With the solution region divided into M x N = 30 x 30 sections， 
what is the value ofy = 4(Anm27/(Az)2? Find the maximum absolute 
error of the Solution obtained by using the routine“wave () ”. 

( 廊 With the solution reglion divided into AM x N = 30 Xx49 Sections， 
what is the value of r?7 Find the maximum absolute error of the 
numerical solution. 

(iii) If we increase N (the number of subintervals along the 1 axlis) to 
51 for better accuracy，what is the value of 7 ?Find the maximum 
absolute error of the numerical solution. 


(iv) What do you think the best value of r 1S? 


827(x 六 927(x， 人 四 
ax2 92 


(C) forO0<xY<10, 0<f<10 (P9.0.7) 


with the initial/boundary conditions 


2 0) 一 (P9.60.8) 


(一 2)(3 一 X) for2<xY<3 
0 elsewhere 


ou(x,1)/otli-=o 三 0， 2U(0,1) 三 0， UVU(10,1 四 三 0 (P9.0.9) 
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(iD With the solution reglion divided into AM x N = 100 x 100 sections， 
what is the value of = 4(AnD2/(Ax)22 

(ii) Noting that the initial condition (P9.0.8) can be Implemented by the 
MATLAB statement as 


>>it0 = inline('(X-2).*(3-X).*x(2<X&Xx<3) ，X' ); 


Solve the PDE (P9.6.7) in the same way as in“solve_wave.m” and 
make a dynamic Picture out of the numerical solution，with the current 
time printed on Screen. Estimate the time when one of the two Separated 
pulses propagating leftwards is reflected and reversed. How about the 
time when the two Separated pulses are reunited? 


9.7 FEM (Finite Element Method) 


In expectation of better accuracy/resolution， modify the program 
“do_fem.m” (in Section 9.0) by appending the following lines 


;-17132 -31/64; -1/2 -17132;-15/132 -31/164 
;17/132 31/64; 1/2 17132; 15132 31/164 


to the last part of the Node array N and replacing the last line of the subreglion 
array S with 


26 32 33j 27 33 34j 28 32 34j 29 35 36 
30 36 37j 31 35 37j 32 33 34j 35 36 37 


This ls equivalent to refining the triangular mesh in the subregions nearest 
to the point charges at (0.3, 0.53) and (一 0.3, 一 0.9) as depicted in Fig. P9.7. 
Plot the new solution obtained by running the modified program. You may 
have to change a statement of the program as followsS. 


f962 = ' (norm([x yl]+[0.5 0.5])<1e-3)-(norm([x y]-[0.5 0.5])<1e-3) ; 


(1/2,17/32) 
-1/2,-7/16: 
) 


(7/16,17/32) 









(9) 


(9/16,17/32) 







(17/32,-31/64) (15/32,-31/64) 


(15/32,31/64) (17/32,31/64) 





(1/2,7/16) 


C91H6-1782) CT2017132) CC716-17/82) CD 


Figure P9.7 ”Refined triangular meshes. 
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9.8 PDEtool: GUI (Graphical User Interface) of MAILAB for Solving PDEs 


(a) Consider the PDE 


8 87 芝 3 
人 


82x? 0X0y 9y” 
(P9.8.1) 
with the boundary conditions 
U(0, y) 一 ye2>， U(, y) = (1 十 y)el+27， 
(P9.8.2) 


U(z, 0) 一 Xer， U(t, H 一 (十 Des+? 


Noting that the field of coefficient c should be flled in as 


o nipte | 。 


in the PDE specification dialog box and the true analytical solution ls 
HU 人 一 人 十 er (P9.8.3) 


use the PDEtool to solve this PDE and fll in Table P9.8.1 with the 
maximum absolute error and the number of nodes together with those of 
Problem 9.2(d) for comparison. 

You can refer to Example 9.8 for the procedure to get the numerical 
value of the maximum absolute error. Notice that the number of nodes is 
the number of columns of p, which is obtained by clicking "Export_ Mesh” 
in the Mesh pull-down menu and then，clicking the OK button in the 
Export dialog box. You can also refer to Example 9.10 for the usage 
of "Adaptive Mesh', but in this case you only have to check the box 
on the left of "Adaptive Mode`” and click the OK button in the “Solve 
Parameters”dialog box opened by clicking “Parameters”in the Solve 
pull-down menu, and then the mesh is adaptively refined every time you 
click the = button in the tool-bar to get the solution. With the box on the 
left of "Adaptive Mode” unchecked in the “9olve Parameters`” dialog box， 


Table P9.8.1 The Maximum Absolute Error and the Number of Nodes 


The Maximum The Number 

Absolute Error of Nodes 
poisson() 1.92530 41 x 41 
PDEtool with Initialize Mesh 0.1914 177 


PDEtool with Refine Mesh 

PDEtool with second Refine Mesh 
PDEtool with Adaptive Mesh 
PDEtool with second Adaptive Mesh 


(b 


(C 


旋 


旋 
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the mesh is nonadaptively refined every time you click “Refine Mesh” 
in the Mesh pull-down menu. You can restore the previous mesh by 
clicking“Undo Mesh Change” in the Mesh pull-down menu. 

Consider the PDE 


人 一 忆 =0 forO0<Y<4 0<y><4 (P9.8.1) 
with the DirichleUVNeumann boundary conditions 

U(0, 7) 一 e 一 cogy，9zGy)/3axl:-4 = 一 esin4 一 ecosy (P9.8.5) 
0U(, y)/0y|y=0 一 cOSXY，0U(X， y)/ay|) -4 一 e cosXY 十 esSin4 (P9.8.0) 
Noting that the true analytical solution ls 


WU, y) 一 ecosX 一 ecoSy (P9.8.7) 


use the PDEtool to solve this PDE and fll in Table P9.8.2 with the 
maximum absolute error and the number of nodes together with those of 
Problem 9.3(g) for comparison 


Consider the PDE 


927(0x，D 作 OCT) 
D 半 22 
90xX2 0 





for0<x<T，0<1f<0.2  (P9.8.8) 
with the initialboundary conditions 

2U(Xz,0) 一 Sin(27)， 2U(0,1) 一 0， 2U(Tr,1) 一 0 (P9.8.9) 
Noting that the true analytical solution ls 


WOc 1 一 Sin(2x)e-& (P9.8.10) 


Table P9.8.2 The Maximum Absolute Error and the Number of Nodes 


The Maximum The Number 

Absolute Error of Nodes 
poisson() 0.2005 21 x 21 
PDEtool with Initialize Mesh 0.3702 177 


PDEtool with Reftine Mesh 

PDEtool with second Refine Mesh 
PDFEtool with Adaptive Mesh 
PDFEtool with second Adaptive Mesh 
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Table P9.8.3 The Maximum Absolute Error and the Number of Nodes 


The Maximum The Number 
Absolute Error of Nodes 
poisson() 7.5462 x 10 一 41 x 101 


PDEtool with Initialize Mesh 
PDEtool with Reftine Mesh 
PDEtool with second Reftine Mesh 


use the PDEtool to solve this PDE and fll in Table P9.8.3 with the 
maximum absolute error and the number of nodes together with those 
obtained with the MATLAB routine “heat_CN() ”in Problem 9.4(c) for 
comparison. In order to do this job, take the following steps. 


(2 


At 


(3 


S 必 


(4 


(5 


Al 


(O) 


(7) 


Click the 口 button in the tool-bar and click-and-drag on the 
graphic region to _ create a rectangular domain.， Then，double- 
click the rectangle to open the Object dialog box and set 
the LefVyBottom/Width/Height to OOpM0.01 to make a long 
rectangular domain， 
(cf) Even ifthe PDEtool is originally designed to deal with only 2-D PDES， 
we can use it to Solve 1-D PDEs like (P9.8.8) by proceeding in this way. 
Click the 0S button in the tool-bar，double-click the upperlower 
boundary segments to set the homogeneous Neumann boundary con- 
dition (5 一 0,q= 王 0) and double-click the lefyright boundary Seg- 
Iments to set the Dirichlet boundary condition (h = 1,T 王 0) as given 
by Eq. (P9.8.9). 
Open the PDE specification dialog box by clicking the PDE button， 
check the box on the left of "Parabolic” as the type of PDE, and set 
its parameters in Eq. (9.3.9) as c 一 2,a 王 0,f=0andd=1 which 
Corresponds to Eq. (P9.8.8). 
Click "Parameters in the Solve pull-down menu to set the time range， 
Say, as 0:0.002:0.2 and to set the initial conditions as Eq. (P9.8.9). 


In the Plot selection dialog box opened by clicking the 雪 but- 
ton，check the box before Height and click the Plot button. If you 
want to plot the Solution graph at a time other than the final time， 
Select the time for plot from {0, 0.002, 0.004, ... ,0.2} in the far- 
right field of the Plot selection dialog box and click the Plot but- 
ton again. 

H 开 you want to see a movie-like dynamic picture of the solution graph， 
check the box before Animation and then click the Plot button in the 
Plot selection dialog box. 

Click "Export_ Mesh in the Mesh pull-down menu, and then click the 
OK button in the Export dialog box to extract the mesh data {p,eyt |. 
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Also click "Export_ Solution ”in the Solve pull-down menu, and then 
click the OK button in the Export dialog box to extract the solution 
u. Now, you can estimate how far the graphical/numerical Solution 
deviates from the true Solution (P9.8.10) by typing the following 
statements into the MAILAB command window: 


>>X = p(1,:)';)yY= p(2,: )'; %5xy coordinates of nodes in columns 
>>tt = 0:0.01:0.2; %time Vector in row 

>>err = Sin(2*X)*exp(-8*tt)-U;i s%qdeviation from true Sol.(P9.8-10) 
>>err_max = max(abs(err)) smaximum absolute error 


(d) Consider the PDE 


2 人 92x(x, 放 


了 7 for0<x<10,0<!L<10 (P9.8.11) 
和 


with the initial/boundary conditions 


ac-23-mtfor2<r<3 3 
xz, 0) 一 人 elsewiere  ， 机 lo 一 0 9.8.12) 
x(0, 门 ==0， udo,D =0 (P9.8.13) 


Use the PDEtool to make a _ dynamic picture out of the solution for 
this PDE and see 让 the result ls about the Same as that obtained in 
Problem 9.6(c) in terms of the time when one of the two Separated pulses 
Propagating leftward is reflected and reversed and the time when the two 
Separated pulses are reunited. 

(cf) Even if the PDEtool is originally designed to solve only 2-D PDESs,， we can 
Solve 1-D PDE like (P9.8.11) by proceeding as follows: 

In the PDE toolbox window, adjust the ranges of the x axis and the 
y axis to [一 0.3 10.3] and [一 0.01 十 0.01], respectively, in the box 
opened by clicking“Axes_Limits” in the Options pull-down menu. 
Click the 口 ] button in the tool-bar and click-and-drag on the graphic 
region to create a long rectangle of domain ranging from xo 一 0 to 
Xf = 10. Then, double-click the rectangle to open the Object dialog 
box and set the LefVyBottom/WidthMHeight to 0/ 一 0.01/10/0.02. 
Click the 0S button in the tool-bar，double-click the upperlower 
boundary segments to set the homogeneous Neumann boundary con- 
dition (5 一 0,q= 三 0) and double-click the lefyright boundary Seg- 
ments to Set the Dirichlet boundary condition (h 和 1,T 王 0) as given 
by Eq. (P9.8.13). 

Open the PDE Specification dialog box by clicking the PDE button， 
check the box on the left of "Hyperbolic” as the type of PDE, and 
Set its parameters in Eq. (P9.8.1D) as c= 王 1a=0,f=0andd=1. 
(See Fig. 9.15a.) 
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(4 


(G) 


〈O) 


(7) 


(3) 


Click "Parameters in the Solve pull-down mennu to set the time range 
to, Say, as 0:0.2:10, the boundary condition as (P9.8.13) and the ini- 
tial conditions as (P9.8.12). (See Fig. 9.1Sb and Problem 9.6(c)(GD).) 


In the Plot selection dialog box opened by clicking the 二 button ， 
check the box before“Height and the box before "Animation ”and 
then click the Plot button in the Plot selection dialog box to see a 
Imovie-like dynamic picture of the solution graph. 

H 开 you want to have better resolution in the solution graph, click Mesh 
in the top menu bar and click “Refine Mesh in the Mesh pull-down 
Imenu. Then, select Plot in the top menu bar or type CTRL 二 P(P) 
on the keyboard and click "Plot Solution in the Plot pull-down menu 
to see a Smoother animation graph. 

In order to estimate the time when one of the two separated pulses 
propagating leftward is reflected and reversed and the time when 
the two Separated pulses are Teunited，count the fickering frame 
numbers, noting that one 全 ckering corresponds to 0.2 S according 
to the time range set in Step (4). 

Hyou want to Save the PDEtool program, click File in the top menu 
bar, click “9ave_ AS in the File pull-down menu, and input the file 
name of your choice. 
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MEAN VALUE THEOREM 





Theorem A.1. Mean Value Theoreml. Let a function F(x) be continuous on 
the interval [a,p] and differentiable over (a, D). Then, there exists at least one 
point 5 between da and D at which 


JJ O) 一 /GO) 


1 加 = 一 一 


JO) = 7OT+T 大 GO 一 0) (A.D 
In other words, the curve of a continuous function 太 (x) has the Same Slope as 
the straight line connecting the two end points (a, ja)) and (2, 帮 ()) of the 
curve at Some point ge [a, Dj], as in Fig. A.1. 














Figure A.1 Mean value theorem. 


1 See the website @http:/www.maths.abdn.ac.uk/~igc/testing/tch/ma2001/notes/motes.html 
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Theorem A.2. Taylor Series Theoreml. If a function FCx) is continuous and 
its derivatives up to order (K 十 J) are also continuous on an open interval 刀 
containing Some point wa, then the value of the function jx) at any point x E 姜 
can be represented by 


KPAK 
JCD = > 万 2 一 0 十 RH1C) (AI) 
K 一 0 


where the first term of the right-hand Side is called the Kth-degree Taylor poly- 
nomlial, and the second term called the remainder (error) term 1s 


FEKTDG) 
( 开 十 1)! 


天 十 1 


及 KTIC) 一 (一 0) for some 上 5 between C andx  (A.3) 


Moreover, if the function 太 (x) has continuous derivatives of all orders on 呈 ， 
then the above representation becomes 





名 大 
Jo= 和 革 9 2 一 中 (人 A. 
KK 一 0 


which is called the (infinite) Taylor series expansion of 太 (x) about a. 
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MATRIX 
OPERATIONS/PROPERTIES 





B.1 ADDITION AND SUBTRACTION 
QI2 到 | DID2 DPIv 
A48| 


QGM1 GaQM2 ， QMN DMI DB PDMN 


C11 Cl12 ”CIN 
C21 C22  ， C2NM 一 C (B.1.D 


CM1  CM2 ” CMN 


with 
Go 十 Dior 一 Comn (B.1.2) 


B.2 MULTIPLICATION 
0 GaI2 QIK ppD2 :DPIv 
4 一 | 021 022 ， 02K | | ppD2 ， pov 
| 而 CQM2 部 | 这 pk2 ， DPKN 
C11 512 C1N | 
三 | 和 | (B.2.1) 
CM1 CM2 ” CMN 
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with 


天 
Cn 一 》 ,amkpn (B.2.2) 
K 一 1 


(cf For this multiplication to be done,， the number of columns of 4 must equal the 
number of rows of 妃 . 


(cf Note that the commnutative law does not hold for the matrix multipllication, that jls， 
4 呈 和 有 4. 


B.3 DETERMINANT 


The deteraizaap1t ofa 天 X 天 (Square) matrix 4 三 [aol defined by 


天 天 
det(4) 一 4 一 2 am 人 (DYMo or > amk(D" Me GB.3.) 
K 一 0 K 一 0 
for any fxed 1<<72<K or 和 几 芝 天 


where the minor Mk jls the determinant of the (KK 一 1) x(K 一 1] (minor) 
matrix formed by removing the kth row and the mth column from 4 and 4 六 一 
(一 DA is called the cofactor of ai， 

In particular， the determinants of a 2x2 matrix 42xz and a 3xXx3 matrix 


43>3 are 
东 








QI11 212 大 十 刀 
det(42x2) 一 区 全 》 ai (一 ID Mo = da11922 一 G12021 (B.3.2) 
大 一 1 
QI1 0412 9013 
222 923 0Q21 023 CQ21 22 
det(43x3) 三 | al1 2 a23 | = al 一 012 十 013 
032 033 0G31 033 031 932 

















031 0432 033 


三 411(022033 一 023032) 一 G12(021933 一 023031) 十 013 (021032 一 022031) 
(B.3.3) 
Note the following properties. 


。If the determinant of a matrix ls zero, the matrix ls Singular. 

。 The determinant of a matrix equals the product of the eigenvalues of a 
IatrIX. 

。 HI 4 1 uppervlower triangular having only zeros below/above the diag- 
onal in each column，its determinant is the product of the diagonal ele- 
Iments. 

。det(47) = det(4); det(4B) = det(4)det(B); det(4-0 = 1/det(4) 
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B.4 EIGENVALUES AND EIGENVECTORS OF A MATRIX2 


The eigemyvalue or characteristic value and its corresponding eige1vector Or char- 
acteristic vector of an NM x N matrix 4 are defined to be a scalar 入 and a nonzero 
Vector V Satisfying 


4v=A 和 My 今 (4-XDy=0Cw 关 0) (B.4.1) 


whpere (人 ,V) ls called an eligenpair and there are N eigenpairs for an NM x N 
matrix 4. 
The eigenvalues of a matrix can be computed as the roots of the characteristic 
equation 
|4 一 入 中 =0 (B.4.2) 


and the eligenvector corresponding to an eligenvalue 入; can be obtained by Sub- 
stituting 和 ; into Eq. (B.4.1) and Solve it for V. 
Note the following properties. 


。 HT 4is Symmetric, all the eligenvalues are Teal-valued. 


。I 4 symmetric and positive deftinite，all the eigenvalues are Teal and 
positive. 
。 HIyis an eligenvector of 4, So is cy for any nonzero Scalar c. 


B.5 INVERSE MATRIX 

The 加 verse matrix of a 玉 x 玉 (Square) matrix 4 一 [ay] is denoted by 4-1 
and defined to be a matrix which is premultiplied/postmultiplied by 4 to form 
an ldentity matrix 一 that is,，Ssatisfies 


4xA4-=A4-x4=7 (B.5.1) 


An element of the inverse matrix 4-1 = [an] can be computed as 


= LTDntnMy (B.5.2) 
Qi 二 一 4 一 一 人 一 7 ) 
det(A) 风 | 


Where Mi is the minor of an and 4 = (一 DAM is the cofactor 
of Cr 


2 See the website @http:/www.sosmath.cony/index.html or http:Wwww.psc.edu/~burkardtpapers/ 
linear_glossary.html.) 
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Note that a square matrix 4 is invertible/nonsingular 寺 and only 革 


。 No eigenvalue of 4 is zero, or equivalently， 
。 The rows (and the columns) of 4 are linearly independent, or equivalently， 
。 The determinant of 4 is nonzero. 


B.6 SYMMETRIC/HERMITIAN MATRIX 
A square matrix 4 is Said to be sy1z11aietjic iit is equal to its transpose, that js， 
47= 三 4 (B.6.1) 


A complex-valued matrix is Said to be 有 ermaifian 让 it ls equal to lts complex 
conjugate transpose, that is， 


4 三 4 where* means the conjugate. (B.0.2) 
Note the following properties of a symmetric/Hermitian matrix. 
。 All the eligenvalues are real. 


。 If all the eigenvalues are distinct，the eligenvectors can form an orthogo- 
nal/unitary matrix L7. 


B.7 ORTHOGONAL/UNITARY MATRIX 


A nonsingular (Square) matrix 4 is Said to be ortpogo1al if its transpose ls equal 
to its inverse, that ilS， 
474 三 了 47 三 47-1 (B.7.1D) 


A complex-valued (Square) matrix is Said to be xmitary 这 its conjugate transpose 
is equal to its inverse, that ls， 


44 三 几 站 和 -三 丰 = (B.7.2) 
Note the following Properties of an orthogonalunitary matrix. 
。 The masgnitude (absolute value) of every elgenvalue ls one， 


。The product of two orthogonal matrices is also orthogonal; (4B)s(4B) 一 
BT7(4T 4) 三 工 
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B.8 PERMUTATION MATRIX 


A matrix 已 having only one nonzero element of value 1 in each row and column 
is called a PetUiation matrix and has the following Properties. 


。Premultiplication/postmultiplication of a matrix 4 by a permutation matriX 
P (ie., P4 or 4P) yields the row/column change of the matrix 4, reSpec- 
tively. 

。A permnutation matrix 4 is orthogonal, that is, 474 三 了 


B.9 RANK 


The ra of an MxN matrx is the number of linearly independent 
IOowSs/columns and if it equals min(M, NW)，then the matrix is Said to be of 
1IQXI1ICL OFT jl rank; otherwise, the matrix 18 Said to be 7a1K-deficiemt or to 
have 7a1K-dejicriemcy. 


B.10 ROW SPACE AND NULL SPACE 


The 7row space of an M x N matrix 4, denoted by 有 尺 (L4), is the space Spanned 
by the row vectors 一 that is, the set of all possible linear combinations of row 
vectors of 4 that can be expressed by 47a with an M-dimensional column vector 
&. On the other hand, the 11 space of the matrix 4, denoted by WCG, is the 
Space orthogonal (perpendicular) to the row Space 一 that is, the set of all possible 
linear combinations of the NW-dimensional vectors Satisfying 4x = 0. 


B.11 ROW ECHELON FORM 
A matrix is Said to be of 7row ecpeloz jozz7 让 


。 Each nonzero Tow having at least one nonzero element has a ]l as its first 
nonzero element. 

。 The leading 1 in arowils in a column to the right of the leading 1 in the 
Upper Tow， 

。All-zero TIowsgs are below the rows that have at least one nonzero element. 


A matrix ls Said to be of 7edwced fow echnelom joz1 让 it Satisfies the above 
conditions and, additionally，each column containing a leading 1 has no other 
nonZzero elements. 
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Any matrix, singular or rectangular, can be transformed into this form through 
the Gaussian elimination procedure (ie., a Series of elementary row operations) 
or, eduivalently, by using the MAILAB built-in routine“rref()”. For example， 
we have 


001 3|  ，[f240 -8 
4=|240 -8| 一 >|001 3 
| 
了 1 2 0 一 4 二 1 2 0 一 4 
0 0 1 3 | 一 一 人 |0 0 1 3 | = rre14) 
subtraction | 0 0 1 3| saiomn |0 00 0 


Once this form is obtained, it ls easy to compnute the rank, the determinant and 
the inverse of the matrix, if only the matrix is invertible. 
B.12 POSITIVE DEFINITENESS 
A square matrix 4 is Said to be Positive dejanite 下 

xs A4X>0 for any nonzero Vector X (B.12.1) 
A square matrix 4 is Said to be Posi1ive Se11atde1izzite 让 

xz4x>0 for any nonzero vector X (B.12.2) 

Note the following properties of a positive definite matrix 4. 


。 4 is nonsingular and all of its eligenvalues are positive， 
。The inverse of 4 is also positive definite. 


There are Similar definitions for megative de1jizazztenessy and egative 8e1111de1i1zite- 
11ES5. 


Note the following property，which can be used to determine if a matrix 


ls positive (Semi-) definite or not. A square matrix ls positive definite 过 and 
only 下 : 


(人 Every diagonal element is posltive. 
(ii Every leading principal minor matrix has positive determinant， 


On the other hand, a Square matrix is positive Semidefinite 让 and only 于: 


(人 Every diagonal element is nonnegative. 
(Ti) Every principal minor matrix has nonnegative determinant. 
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Note also that the Principal minor matrices are the submatrices taking the diagonal 
elements from the diagonal of the matrix 4 and, Say for a 3 x 3 matrix，the 
Principal minor matrices are 


Q11 212 022 023 QI11 413 疙 1 和 1 
oo 本 | 加 | 021 022 023 


among which the leading ones are 


B.13 SCALAR (DOT) PRODUCT AND VECTOR (CROSS) PRODUCT 


Ascalar product of two N-dimensional vectors X and y ls denoted byX.y and 


is defined by 
N 


x.y 了 一 》 mm 六 一 xy (B.13.1) 


二 


An outer pioduct of two three-dimensional column vectors X 一 [xl x2 X3]7 and 
y 一 [yy 轨 ]Tis denoted byxxyandis defined by 


X27y3 一 XY3yY2 
入 XyY 王 | X3yl 一 XYl1yY3 (B.13.2) 
X17yY2 一 XY2yY1 


B.14 MATRIX INVERSION LEMMA 


Matrix Inversion Lemma. Let 4, C, and [C-1 十 D4-1B]be well-defned with 
nonsingularity as well as compatible dimensions. Then we have 





[4+BCD]-L=A4-1--A4-1BIC-1+DA4-1B]-1D4-1 (B.14.1) 


Pioof We will show that postmultiplying Eq. (B.14.1) by [4 十 BCD]yields an 
identity matrix. 





[4- -4-!B[C-I++D4-1B]-D4-I4+BCD] 
==7T+A4-IBCD-4-1B[C-+D4-1B]-1D 
-4-1B[C-1 +D4-1B]-1D4-1BCD 
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=7T+4-IBCD-4-IB[C-1+D4-1B]-IC-ICD 

一 4-1B[C-1+D4-1B]-1D4-1BCD 
==7T+4-1IBCD-4-1B[C-1+D4-1B]-1IC-1+D4-IB]CD 
=T+4-BCD-A4-IBCD=7 








APPENDIX @ 


DIFFERENTIATION WITH 
RESPECT TO AVECTOR 





The first derivative of a scalar-valued function 三 X) with respect to a vector 
X 一 [xl xz]7 is called the gradient of Fx) and defined as 


d 
vfom= 和 Jeo= | CD) 


Based on this deftinition，we can write the following equation， 


9 9 0 y1 
兰 学 一 一 一 三 人 过 
了 XY 一 芭 7 X 0 十 X2y2) 区 y 〈《C.2) 
9 9 ，2 2 X1 | 
二 < 二 + 动 =2| 2 三 X (C.3) 
Also with an M x N matrnx 4, we have 
人 
X 4y 王 一 4X=4y (C.4a) 
OX OX 
0 9 
一 4x= 一 xX747y = 47y (C.4b) 
OX OX 


where 


M N 
X7A4y iT 》， >》， Cj yz (C.3) 


1 一 ] 7 一 1 
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472 ”DIFFERENTIATION WITH RESPECT TO A VECTOR 
Especially for a square, Symmetric matrix 4 with M = NW,， we have 


0 玉 丰 让 4is symmetric 
了 4xX = (4 二 4 )X 一 > 24x (C.0) 
玉 


The second derivative of a scalar function 太 X) with respect to a vector X 一 
[xi xz]7 is called the Hessian of Fox and is defined as 


dd 827/3axz 3827/3xl197x 
刀 (x) 一 V? ES 三 C.7 
人 |> /amaxl 382/3x2 4 
Based on this definition，we can wiite the following equation: 
人 2 7 7 让 4 is symmetric 
一 一 X 4x = 4 二 4 一 人 > 24 (C.8) 


CX2 


On the other hand, the first derivative of a vector-valued function fx) with 
Tespect to a vector X 一 [xl xz2]7 is called the Jacobian of 太 (x) and is defined as 


四 | 967/ox 907/0x2 
Jo 和 一 | 纹 全 | 4 


APPENDIX D 


LAPLACE TRANSFORM 





Table D.1 Laplace Transforms of Basic Functions 














X(1) X(y) X(1) X(9) X(1) 
(]) 6(0D) 工 ($) eps () 2 (9) e- 江 Sincf xy(D) 
3 十 Q 

一 刀 8 1 一 一 G8 11 一 CE 

(2) 0(L 一 帮 ) 放 (0) 区 2 (T) GO (10) e COS COf sy (T) 
1 CO 

G) (D -OO sinowriO 
史 十 

本 111 
mnD NT (Geosolm() 一 
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Table D.2 ”Properties of Laplace Transform 





(0) Deftinition X(s) 三 ZL{x(D} 到 人 roed 
0 
(D) Linearity ax(D) 十 Bx() 一 aX(G) 十 B7CG) 
0 
C) Time shifting XG 一 1 人 JUsC 一 四 二 >0 一 6 lxe + X(T)e | 


(3) Frequency shifting 。 ex 人 ) 一 X(G 一 1) 
(4) Real convolution gxx() 一 G(GC)X(C) 


(3) Time derivative X'1) 一 SSX() 一 X(0) 
芝 1 1 0 
(60) Time integral / X(T)dr 一 -X() 十 = X(T)QT 
二 3 3 wo 


d 
(7) Complex derivative 1x() 一 二 (8) 
8 


1 0a0 十 co 
(8) Complex convolution x(f)y() 一 元 上 XU7(Gs 一 vdv 
TJ] Ja 


OO: 


(9) Initial value theorem x(0) 一 lm 8sX(y) 
3 一 OO 


(10) Final value theorem x(co) 一 im SX (8) 


APPENDIX E 


FOURIER [TRANSFORM 





Table E.1 Properties of CtFT (Continuous-Time Fourier Transform) 


(0) Deftinition 


(1) Linearity 
(2) Symmetry 


(3) Time shifting 
(4) Frequency Shifting 


(9$) Real convolution 
(6) Time derivative 


(7) Time integral 
(8) Complex derivative 
(9) Complex convolution 


(10) Scaling 
(11) Duality 


(12) Parseval's relation 


Co 


X(w) = reol= 上 Xe 一 7ox di 


ax(f) 十 Bx0) 一 QXwO) 十 BF7(w) 

X(1) 一 Xe(f) 十 Xo(: Teal 一 X(w) 三 X( 一 O 〇 ) 
Xe(1): TIeal and even 一 Xe(w) = RefX(w)} 
xXo(1): real and odd 一 Xo(wo) = JImf{X(w)} 
X( 一 1) 一 X( 一 〇 ) 

XL 一 万 ) 一 e-7/onX(w) 


eyolx() 一 X(wO 一 ol) 





Co 


8g(i)#kx(i) 一 1 grT)xL 一 rz)dr 一 COX(w) 


0 


xD) 一 JoX(w) 


1 x(r)dr 全 工 Xoj 上 TrX(Oid) 
汪 JW 


Co 


2 
aow 
X(1)y() 一 了 工 X(w) 洲 了 (@w) 
2 元 


X(a1) 一 二 文思 玉 
al 


80) 一 jw) 今 /0 一 2r8(w) 
1 |x(GD)|2 dz 一 云 上 xz(o)| 2 do 
区 2 大 


Co 
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476 FOURIER TRANSFORM 


Table E.2 Properties of DtFT (Discrete-Time Fourier Transformm) 


(0) Definition 


(1) Linearity 
(2) Symmetry 


(3) Time Shifting 
(4) Frequency shifting 


($) Real convolution 


(6) Complex derivative 
(7) Complex convolution 


(8) Scaling 


(9) Parseval s relation 


Oo 


X(Q) = >》， X[z]e-79n 


coxX[1] 十 Bx[] 一 wwXC) 十 BF7(S9) 

X[7] 三 Xe[7] 十 xzo[2]: real 一 X() 三 X (一 O) 
Xe[m]: real and even 一 Xeo(O) 一 Re(X(CSO)} 
Xo[m]: real and odd 一 Xo(S) = JIm{X(C)} 
X[ 一 7] 一 X( 一 2) 

XI 一 1 一 emX(OD) 


ejonx[] 一 XCQ 一 QI) 
Ooe 


8g[m]#x[g= 》，8Umxm 一 四 一 GOOD)X(CO) 


1 一 一 Co 


Q 
一 -<X (2 
.XIz] 一 7 7 (9) 
1 
X[m2]y[2] 一 5 人 (2) * 了 (22) (periodic/circular convolution) 
元 


~ X(OMOD) 


X[2/AM] 这 =1AMO0 :an integer) 
0， otherwise 


> emP= 去 人 roPae 
2 元 2 区 


及 一 一 CO 


APPENDIX F 


USEFUL FORMULAs 





下 O17IULaS Jor SU12111G1io OF Fi7ite Nu11IDer Oo 7T217118 














N N 
1 一 ZXN+1i 1 一 (CN+DUDax 二 Nax+l 
3 ae 

2 = 一 (GE.D 和 记 Q 人 (G.2) 
NOV+D) NOV+TDON+D) 
>》 0 (F.3) >， 人 (F.4) 

2 6 
1 一 0 7 一 0 
N 

NO 上 DOX+2 

> no 十 1 = 2 (F.5) 
17 一 0 攻 

芝 NP Ni 

N_ 人 一 由 77 。 人 E 天 ,二 
(二 D)V = 2 NGC wih NGC = NCw- 一 1 (GE.0) 
EO17T1IULGS Jr SU1111G1iO OF Tife NIDer of 7T277118 
Eee 1 芋 大 
区 坑 = 一， lx| < 1 CE.7) 7 站 lxzl| <1 (GF.8) 
1 一 0 7 一 0 
co 大 
0 区 
和 人 
7 = lim(-D 去 | 二 二 lxzl <1 GF.9) 
污 全 十 十 (GF.10) 
一 en 3 em “一 一 丈 上 

全 22+1 全 人 4 


(co721i7Ued oyveried 门 
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478 USEFUL FORMULAS 











大 亡 1 刀 1 1 1 2 1 还 
过 人 
放 二 才 


























Do 
(no)" Ina no” ， (no3 3 
人 二 玉 人 
和 有 区 
TREE 
s 1 1 1 
人 士 X 本 士 忆 区 ,tl <1 
Oo 
， (一 ])” 21 十 1 1 3 1 5 1 
1 二 这 二 的 十 可 一 下 十 …: 
Oo 
(1 27 1 和 1 4 1 6 
2 人 
予 间 3 上 过 5 上 | 元 
anX 一 X 十 一 X ga 人 
3 15 2 


T7z8go7o11aetric FoFmatlas 


Sin(4 十 有) 一 Sin4cos 有 十 cogs4Sn 刀 (GEF.19) tan(4 士 已) 一 


coS(4 士 有 B)=cos4cos 有 二 sn4sn 刀 (FE.20) 




















1 
Sin4sn 有 一 7{cos(4 一 恺 ) 一 cos(4 十 也 )} 
. 1 ，， ， 
Sin 4cos 刀 一 7 了 fsin(4 十 下 ) 十 Sin(4 一 再)} 
， ， 
coSs 4sin 妨 一 fsin(4 十 ) 一 Sin(4 一 已 )} 
1 
coS 4cos 号 = fcos(4 十 已 ) 十 cos(4 一 已)} 
: ， (4 二 了 4 一 了 
Sn4 十 Sn 中 一 2Sin COS 
和 作 
4 十 了 8 4 一 卫 
coS 4 十 cos 姬 一 2cos 7 COS 7 


忆 

acos4 一 psin4=Va2 十 bcos(4 十 0),0 =tan-l (2 
G 
忆 

asin4 十 pcos4=Va2 十 bsin(4 十 0),0 =tanr-l (2 
G 


1 
sin2 4 = 7 一 cogs24) 





tan 4 十 tan 尼 





1 二 ttn4tan 刀 


1 
(FE.30) cos 4 = 70 十 cos24) 


(GEF.1D) 


(F.12) 


(F.13) 


(F.14) 


(F.15) 


(F.160) 


(F.17) 


(F.18) 


(FE.21) 


(F.22) 


(F.23) 


(F.24) 


(F.25) 


(F.26) 


(F.27) 


(F.28) 


(F.29) 


(FE.31) 
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1 1 
sin3 4 一 749 SnA4 一 Sn34) (F.32) cos3 4 一 9 cogs 4 十 cos34) (F.33) 
Sin24 一 2Sin4cosA4 (上 .34) sin34 =3sin4 一 4sin3A4 (.39) 
cos24=cos24-sn24=1-2sin24=2cos24 一 1 (F.360) 
cos34 一 4cosy 4 一 3sin4 (人 .37) 
Q 忆 C 十 各 总 
人 (E.38) e- 一 cos0 十 JSsin0 (F.40) 
Sin 4 Sin 已 Sin C 
3 
d2 一 刀 十 co 一 2bccos4 (FF.39a) sin 0 一 5 一 e-70) (GE.41a) 
J 
1 本 有 ， 
2 一 c? 十 0 一 2cacosB (FEF.39b) cos60 一 宁 2 十 e-70) (.41b) 
1 e718 一 e- 放 
cz 一 民 十 刀 一 2apcosC (FE.39c) tang = 一 一 (GE.41c) 





APPENDIX G 


SYMBOLIC COMPUTATION 





G.1 HOW TO DECLARE SYMBOLIC VARIABLES AND HANDLE 
SYMBOLIC EXPRESSIONS 


To declare any variable(S) as a Symbolic variable，you Should use the sym or 
syms command as below. 


>>a = Sym 


( It= Sym(t');Xx= Sym( Xx' ); 
>>SymsS a X 0 


a ) 
y ts%or，edquivalently and more efficientJy 

Once the variables have been declared as Symbolic, they can be used in expres- 
Sions and as arguments to many functions without being evaluated as numeric. 


>>f = X^2/1(1 + tan(Xx)^2) ; 
>>ezp1lot(f, -pi,pi) 
>>Simp1lify(cos(Xx)^2+Sin(X)^2) %simplify an expression 
ans = 1 
>>Simplify(cos(x)^2 - 
ans = 2*xcos(Xx)^2-1 
>>Simple(cos(X)^2 - Sin(Xx)^2) %simp1e expression 
ans = Cos(2*X) 
>>Simple(cos(X) + ix*Ssin(Xx)) %5Ssimp1e expression 
ans = exp(Ii*X) 
>>eq1 = expand((x +y)^3 - (x + y)^2) s%5expand 
eq1 = X^3 + 3*X^2x*y + 3*Xx*y^2 + yY^3 - X^2 - 2*Xxx*y - y^2 
>>CO1L1ect(edq1,y) %collect Similar terms in descending order with respect to y 
ans =y?3+ (3*X - 1)*y^2 + (3*Xx^2 - 2*Xx)*y + X^3 - X^2 


Sin(X)^2) %5Ssimplify an expression 
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>>factor(edq1) %factorize 
anS= (X+y- 1)*(x+y)^2 
>>horner(edq1) %nested multip1lication form 
ans = (-1+y)*y"2+((- 2+83*y)sy+ (TY+S3sy + X)sX)*X 
>>pretty(ans) %pretty form 
2 
(-1+y)y +((2+3y)y+ (1+3y+X) X) X 


H 开 you need to Substitute numeric values or other expresslions for Some Sym- 
bolic variables in an expression, you can use the subs function as below. 


>>Subs(eq1,Xx,0) %Substitute numeric value 
ans = -y^2 + Yy^3 

>>Subs(eq1,{x,y},{0,x - 1}) #ssubstitute numeric Values 
ansS= (X-1)^3- (x -1)^2 


The sym command allows you to declare Symbolic real variables by using the 
Teal option as illustrated below. 


>>X = Sym(' X' ，real' )jy= Sym( y' ，real' ); 

>>Syms X yY real %50r，edquivalentJy 

>>Z = X + Ixyj %qdqeclare Z as a Symbolic compJex Variab1l1e 
>>Cconj(z) %complex conjugate 


ans = X - ixy 
>>abs(Z) 
ans = (x^2 + y^2)^(1/2) %equivalent1y 


The sym function can be used to convert numeric values into their Symbolic 
eXpresSlons. 


>>Sym(1/2) + 0.2 
ans = 7/10 %5symbolic expression 


On the other hand, the double command converts Symbolic expressions into 
their numeric (double-precision foating-pointb) values and the vpa command finds 
the variable-precision arithmetic (VPA) expression (as a Symbolic representation) 
of a numeric or Symbolic expression with d Significant decimal digits，wphere d 
is the current Setting of DIGITS that can be set by the digits command. Note 
that the output of the vpa command is a Symbolic expresSion even 寺 it may look 
like a numeric value. Let us See Some examples. 


>>f = Sym('exp(i*pi/4) ) 
f = exp(ix*pi/4) 
>>double(f) 
ans = 0.7071 + 0.70711 %numeric value 
>>Vvpa(ans,2) 
ans = .71 + .71*i %5Symbolic expression with 2 Significant digits 


CALCULUS 483 
G.2 CALCULUS 


G.2.1 Symbolic Summation 


We can use the symsum() function to obtain the sum of an indefinite/definite 
Series as below. 


>>Syms X n N %5declare xn;N as Symbolic variab1les 
>>Simple(Symsum(n,0,N) ) 


NOV 上 + 1 
ans = 1/2xN*(N + 1) %y 7 一 2 


2 
>>Simple(Symsum(n^2,0,N) ) 


NON ])CN 工 
ans = 1/6xNx(N + 1)*(2x*xN + 1) %y 72 二 CV 二 JU 十 J) 








0 
>>Symsum(1/n^2,1,inf)) 
ans = 1/6*pi^2 ws 一 本 
用 0 
>>SymSsum(x^nynyoinf)) 
ans = -1/(-1 + X) 妥 史 人 下 二 ] under the assumption that |x| < 1 


G.2.2 Limits 


We can use the Limit() function to get the (two-sided) limit and the TighUleft- 
Sided limits of a function as below. 


>>Syms h n X 





. Sin X 
>>1imit(sin(X)/Xx,Xx,0) % lim 三 
X 一 0 Xi 
ans = 1 
X 
>>1imit(x/abs(Xx),Xx,0，right') %lm 一 一 1 
一 0+ | 2 
ans = 1 
X 
>>1imit(x/abs(X),Xx,0，1left') %5lim 一 一 一 | 
基 - 人 0 |x| 
ans = -1 
>>1imit(x/abs(Xx),Xx;,0) % lm 一 一 7? 
x 一 0 |z| 
ans = NaN  %Not a Number 
网 县 . cosG 十 有) 一 cos(r) Q . 
>>1Limit((cos(x+h) -cos(x) ) /hh,0) 5 lm 一 = 一 COSY 一 一 Sinx 
矿 一 0 己 ax 


ans = -Sin(X) 


并 交 二 
>>1imit((1 + x/n)^nnyinf) 5 lm 4 十 =) 6 
天 一 OO 


ans = exp(Xx) 


G.2.3 Differentiation 


The diff() function differentiates a Symbolic expression w.I.t. the variable given 
as one of its 2nd or 3"d input arguments or its free variable which might be 
determined by using the findsym function. 
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>>SymSs ab xn 
>>diff(x^n) ) 
ans = X^nxny/X 
>>SimpJify(ans ) 
ans = X^(n - 1)*n 
>>f = exp(axX)*xcos(bxt) 
>>diff(f) %equivalently diff(f,X) 


QZ 
ans = axexp(a*X)*Ccos(bxt) 5% 一 厂 一 一 ecos (D1) 一 ae 和 cos (D1) 
ax ax 
>>diff(f,t) 
ans = -exp(a*Xx)*Sin(bx*t)*b 5 三 元 8” cos 01 一 一 pe Sin (D1) 


>>diff(f,2) %edquivalent1Jly diff(f,x,2) 
2 


太一 da2ew cos (D1) 





ans = a^2*eXxp(ax*X)*CoS(bx*t+) 55 
Cx2 


>>diff(f,t,2) 
2 


dl7 





ans = -exp(axx)x*xcos(bxt)x*b^2 % 一 一 一 ecos (D1)D2 
>>g = [cos(X)*cos(t) cos(Xx)*Sin(t)]; 
>>jacob g = jacobian(g,[x t]) 
jacob g= [ -Sin(Xx)*cos(t)，-cos(x)*xSsin(t)] 
[ -Sin(x)*Ssin(t)， cos(Xx)*xcos(t)] 


Note that the jacobian() function fnds the jacobian defitned by (C.9) 一 that is， 
the derivative of a vector function [gl 8g?]7 with respect to a vector variable 


[如 一 as 
| 98gHMox 081/81 
和 上 全 


G.2.4 Integration 


The int() function returns the indefinite/definite integral (anti-derivative) of a 
function or an expression with respect to the variable given as its Second input 
argument or its free variable which might be determined by using the findsynm 
function.， 


>>SymS a X y 七 





>>int(x^n) 
上 
ans= xc(n+1 n+1)s%hxzcdx = 一 x+l 
人 )/( ) %/ 
>>int(1/(1 + Xx^2)) 
ans = atan(x) 到 dx = tan 1x 
>>int(a^Xx) %5equivalently diff(f,x,2) 
ans = 1/1og(a)*a^"x 5%jax dx = at 
log 4a 


>>int(Sin(axt),0,pi) %5equivalent1ly int(Sin(axt),t,0,pi) 


区 


1 
ans = -cos(pixa)/a + 1/ashj sin()d = 一 二 cos (on = 一 一 cos (or ) 十 二 
da 5 Q Q 
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>>int(exp(-(Xx - a)^2),a,inf) #%5equivalently int(exp(-(Xx - a)^2),x;0,inf) 


ans = 1/2x*pi*(1/2) ss /eco wx= ie d = 二 V 


G.2.5 _ Taylor Series Expansion 


We can use the taylor() function to find the Taylor series expansion of a 
function Or an expresSion with respect to the variable given as jlts Second or 
third Input argument or its free variable that might be determined by using the 
findsym function. 

One may put “help taylor” into the MAILAB command window to see its 
Usage，which is restated below. Let us try applying 1it， 


>>syms x tj N = 3 
1 
>>Tx0 = taylor(exp(-x)N + 1) 叉 C) 二 > _， 二 (0) me 
灵 . 
Tx0 = 1-x + 1/2x*x^2 - 1/6xx^3 


>>Sym2poly(Tx0) %sextract the coefficients of Taylor series polynomial 
ans = -0.1667 0.5000 -1.0000 1.0000 


区 N 1 
>>xo = 1; Txl = taylor(exp(-x),N + 1;xo) ff) = > -0 二 7 oo) GE 一 20 
Tx1 = exp(-1) - exp(-1)*(Xx - 1) + 1/2*exp(-1)*(X - 1)^2 - 1/6*exp(-1)*(X - 1)^3 


>>pretty(Tx1) 
2 3 
exp(-1) -exp(-1)(Xx - 1) +1/2 exp(-1)(x - 1) -1/6 exp(-1)(x - 1) 


>>f = exp(-X)*Ssin(t) ; 


1 
>>Tt = taylor(fN + 1it) 台 0) 二 2 -0 /OP 
玫 . 


Tt = exp(-X)*t - 1/6*exp(-X)*t^3 





。 taylor(f) gives the fifth-order Maclaurin series expansion of 熙 . 

。 taylor(f,n+1) with an integern > 0 gives the nth-order Maclaurin series 
expansion of 车. 

。taylor(f,a) with a real number (a) gives the fifth-order Taylor series 
expansion of f about a. 

。 taylor(f,n + 1,a) gives the nth-order Taylor series expanslion of f about 
default_variable=a. 

。 taylor(f,n + 1，aiy) gives the nth-order Taylor series expansion of f(y) 
about y = a. 


(cf) The target function f must be a legitimate expression given directly as the first 
input argument. 

(cf) Before using the command “taylor()”, one Should declare the arguments of the 
function as Symbols by putting, Say， “syms X t”. 

(cf) In case the function has Several arguments, it is a good practice to put the inde- 
pendent variable as the last input argument of "taylor()” ,though taylor() takes 
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one closest (alphabetically) to“x” as the independent variable by default only 下 
it has been declared as a Symbolic variable and is contained as an input argument 
of the function 直 . 


(cf) One should use the MATLAB command “sym2poly()” 于 he wants to extract the 
coeffticients from the Taylor series expansion obtained as a Symbolic expresslion. 





G.3 LINEAR ALGEBRA 


Several MAILAB commands and functions can be used to manipulate the vec- 
tors or matrices conslisting of Symbolic expresSlions as well as those consisting 
of numerics， 


>>Syms al11 al12 a21 a22 
>>A = [all al2; a21 a22]; 
>>det(A) 

ans = al1*a22 - a12*a21 
>>AIT =A^ -1 


AI = [ a22/(a11*a22 - al2x*a21)，-a12/(a11x*a22 - al2xa21)] 
[ -a21/(al1xa22 - al2x*a21)， al11/(a11x*a22 - al2x*a21)] 
>>AxAI 
ans = [ al1x*a221/(a11x*a22 - al2x*a21)-al2x*xa21/(a11x*a22 - al2*a21)， 0] 


[ 0，a11*a22/1(al11*a22 - al12*a21) - al2*a21/(a11*a22 - al2*a21)] 
>>Simplify(ans) %Ssimplify an expression 
ans= [1，0|] 
[ 0，1] 
>>Syms X 七 ; 
>>G = [cos(t) Sin(t); -Sin(t) cos(t)] s#The Givens transformation matrix 
G=[ cos(t)，sin(t)] 
[ -sin(t)， cos(t)] 
>>det(G) ，Simple(ans) 
ans = cos(t)^2 + Sin(t)^2 


ans = 1 
>>G2 = G*2，Simp1e(G2) 
G2 = [cos(t)^2 - Sin(t)^2， 2*cos(t)*sin(t)] 
[ -2*cos(t)*sin(t)，cos(t)^2 - Sin(t)^2] 


anSsS=[ cos(2*t)， Sin(2x*t)] 
[ -sin(2x*t)， cos(2*t)] 
>>GTG = G.'*G，Simp1le(GTG) 
[ cos(t)^2 + Sin(t)^2， 0] 
[ 0，cos(t)^2 + Sin(t)^2] 
[ 


[ 0，1] 
>>Simple(G^ - 1) %inv(G) for the inverse of Givens transformation matrIixX 
全 cos(t)，-sin(t)] 
Sin(t)， cos(t)] 
>>Syms b C 
>>A = [01; -c -b]; 
>>[V,E] = eig(A) 
V = -(1/2*b + 1/2*(b^2 - 4*cC)^(1/2))/c，-(1/2*b - 1/2*(b^2 - 4*c)^(1/2))/7c] 
下 1] 
E = -1/2*b + 1/2*(b*2 - 4*c)^(1/2)， 0] 
0 :1F2sb = 于 ww(B 2 si tTF2y] 
>> SOlve(poly(A) )%another way to get eigenvalues(characteristic roots) 
ans = [ -1/2*b+1/2*(b^*2 - 4*c)^(1/2)] 
[ -1/2*b-1/2*(b*2 - 4*c)^(1/2)] 
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Besides，other MATITLAB functions such as jordan(A) and svd(A) can be 
used to get the Jordan canonical form together with the corresponding Similarity 
transformation matrix and the Singular value decomposition of a Symbolic matrix. 


G.4 _ SOLVING ALGEBRAIC EQUATIONS 


We can use the backslash () operator to Solve a set of linear equations written 
in a matrix-vector form. 


>>Syms R11 R12 R21 R22 b1 b2 
>>R = [R11 R12; R21 R22]; b = [b1; b2]; 
>>X = RN\b 
x=[ (R12*b2 - bl1*R22)/(-R11*R22 + R21*R12)] 
[ (-R11*b2 + R21*b1)/(-R11*R22 + R21*R12)] 


We can also use the MAILAB function solve() to Solve Symbolic algebraic 
equations. 


>>Syms a b C X 
>>fX = axX^2+b*X+Cci 
>>Solve(fx) %formu1la for roots of 2nd-order polynomial ed 

ans = [ 1/2/a*(-b + (b^*2 - 4*axc)^(1/2))] 

[ 1/2/a*(-b - (b?*2 - 4*axc)^(1/2))] 

>>Syms X1 x2 bl1 b2 
>>fxl1 = xl1 + Xx2 - bl; fx2 = xl + 2x*x2 - b2; %5a System of Simultaneous algebraic eq . 
>>[Xx10,Xx20] = Solve(fx1,fx2) 5 

Xx10 = 2*b1 - b2 

X20 三 上 1 + 2 


G.5 _ SOLVING DIFFERENTIAL EQUATIONS 


We can use the MAILAB function dsolve() to solve Symbolic differential 
equations. 


>>Syms a b C X 

>>X0 = dsolve('Dx+ arXx= 0') 点 a differential eq.(d.e.) WwW/o initial condition 
Xo = exp(-axt)*Cl 55 a SoOlution with undetermined constant 

>>X0 = dsolve('Dx + arsXx = 0'，x(0) =2') 扣 ad.e. With initial condition 
Xxo = 2*exp(-ax*t) 和 5 a SoOlution with undetermined constant 

>>X0 = dsolve('Dx=1+x^2') %5a differential eq. WwW/0o initial condition 
xo=tan(t - Cl) 点 a solution with undetermined constant 

>>X0 = dsolve('DXx= 1 + Xx*2'，x(0) =1') 5With the initial condition 
xo = tan(t + 1/4*pi) 各 a Solution with determined constant 

>>y0 = dsolve('D2u = -ut')%a2nd-order d.e. without initial condition 
yo = Cl1*sin(t) + C2*cos(t) 

>>X0o = dsolve('D2u = -uU，uU(0) = 1,Du(0) 
Xxo = cos(t)) 

>>y0 = dsolve('(Dy)^2 +y*2 = 1'，y(0) =0'，x')s%alst-order nonlinear d.e.(nlde) 
yo=[ sin(x)] s%two Solutions 

[ -Sin(x)] 

>>y0 = dsolve('D2y = cos(2x*x) - y'i'y(0) = 1;,Dy(0) = 0'，'x')%a2md-order nlde 
yo = 4/3*coS(X) - 2/3*cos(xX)^2 + 1/3 

>>S = dsolve('Df=3*f + 4*g ，Dg=-4*f + 3*g ); 


5 


5 With the initial condition 


1 
口 
十 
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7 
f = exp(3x*t)*(C1x*Sin(4*t) + C2x*CoS(4*t) ) 
g = exp(3*t)*(C1x*cos(4*x*t) - C2x*Sin(4x*t) ) 

>>[f,g] = dsolve('Df = 3*f + 4*g;Dg = -4*f + 3*g ，f(0) = 0,g(0) = 1 ) 
f = exp(3x*t)*Sin(4x*t) 

g = exp(3x*t)*cos(4x*t) 


APPENDIX 站 


SPARSE MATRICES 





A matrix is Said to be Sparse 寺 it has a large portion of zero elements. MAILAB 
has Some built-in functions/routines that enable us to exploit the Sparsity of a 
matrix for computational efficiency. 

The MAILAB routine sparse() can be used to convert a (regular) matrix 
into a Sparse form by squeezing out any Zero elements and to generate a Sparse 
matrix having the elements of a vector given together with the Tow/column index 
vectors. On the other hand, the MAITLAB routine ful11() can be used to convert 
amatrix of sparse form into a regular one. 


>>row_index = [1 12 3 4]; col index = [12234]; elements = [12345]; 
>>m=4;n=4;i As= Sparse(row index，col index,elements mn) 


As = (1,1) 1 
(1,2) 2 
(2,2) 3 
(3,3) 4 
(4,4) 5 
>>Af = full(As) 
Af= 1 2 0 0 
0 3 0 0 
0 0 4 0 
0 0 0 5 


We can use the MAITLAB routine sprandn(m,n;,nzd) to generate an m xn 
Sparse Imatrix having the given non-zero density nzd. Let us See how efficient 
the operations can be on the matrices in Sparse forms. 


>>AS sprandn(10,10,0.2); s%5a Sparse matrix and 
>>Af = ful1L(As); its ful1 version 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang, Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 人 儿 Sons, Inc. 
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>>fJlops(0)，AsA = As*As;i flops %in Sparse forms 
ans = 50 
>>fJlops(0)，AfA = Afx*Af; flops s%in full(regular) forms 
ans = 2000 
>>b = ones(10,1); flops(0)，Xx = As\bj flops 
ans = 160 
>>f1lops(0)，X = Af\b; flops 
ans = 592 
>>fJlops(0)，inv(As); flops 
ans = 207 
>>fJlops(0)，inv(Af); flops 
ans = 592 
>>f1lops(0)，[L,U,P] = Lu(As); flops 
ans = 53 
>>f1lops(0)，[L,U,P] = Lu(Af); flops 
ans = 92 


Additionally, the MAITLAB routine speye(n) ls used to generate an n xn 
identity matrix and the MAILAB routine spy(n) ls used to visualize the Sparsity 
pattern. The computational efficiency of LU factorization can be upgraded 下 
one pre-orders the Sparse matrix by the Symmetric minimum degree permutation， 
which is cast into the MAILAB routine symmmd ( ) . 

JInterest Teaders are welcome to run the following program“do_sparse”to 
fsgure out the functions of Several Sparsity-related MAILAB routines. 


%do_sparse 

Clear，CLf 

%create a Sparse mxn random matrix 

mnm=4jn=5jA=sprandn(mn,，.2) 

%create a Sparse Symmetric nxn random matrix With non-zero density nzd 
nzd = 0.2;j A2 = sprandsym(n;nzd) 

%create a Sparse Symmetric random nxn matrix With condition number 『P 
=0.1; As = sprandsym(nnzd,r) 

#%a Sparse Symmetric random nxn matrix with the set of eigenvalues eigs 
eligs = [0.1 .0.2 .3 .4 .5]; A4=sprandsym(n,nzd,eigs) 

elig(A4) 

tic，A1IA = A1*A1 ，time_Ssparse = toc 

A1f = ful1L(A1); tic，AfAf = A1fx*A1f ' ; time_fu1l1L = toc 

Spy(A1A) ，fu]l1L(A1A) ，A1ATf 

Sparse(A1ATf) 

n=10;A5= sprandsym(n,nzd) 

tic，[L,U,P] = Lu(A5); time_JLuU = toc 

tic，[L,U,P] = LuU(ful1L(A5)); time_ful1 = toc 

mdo = Symmmd(A5); s%5Ssymmetric minimum degree permutation 

tic，[L,U,P] = Lu(A5(mdo,mdo)); tme_md=toc 





(cf) The command "fops” is not available in MAITLAB of version 6.x and that is why we 
Use 'tic” and "toc'” to count the process time instead of the number of foating-point 
operations， 
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MATLAB 





First of all, the following should be noted: 


1. The index of an array in MAILAB starts from 1, not 0. 


2. A dot(.) must be put before an operator to make a termwise (element-by- 
element) operation. 


Some of useful MAITLAB commands are listed in Table 工 .1. 


Table 1.1 Commonly Used Commands and Functions in MATLAB 


Ce1eral Co7z11101205 


break to exit from a for or while loop 
fprintf fprintt( An xX(%d) = %0.4f \a ,ind,x(ind)) 
keyboard Stop execution until the user types any key 
Teturn terminate a Toutine and go back to the calling routine 
]oad 闪闪 水 X Y Tead the values of x and y from the MAILAB fle 
洲 兴 汶 .IOat 
load x.dat Tead the value(s) of x from the ASCII fle x.dat 
SaVe 米 洲 X Save the values of x and y into the MAILAB file 
举 兴 兴 .IOat 
Save X.dat X Save the value(s) of x into the ASCII file x.dat 
clear remove all or Some variables/functions from memory 


7iyo-Dizze1sional GrapPic Co11111G11CS 


bar(x,y),plot(x,y),stairs(X,y) plot the values of y versus x in a barN\continuous 
Stem(x,y),loglog(x,y) NSstairsN\discrete\Xy-log\Xx-log\y-log graph 
Semjlogx(Xx,y),Semilogy(X,y) 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang,，Cao, Chung, and Morris 
Copyright @ 2005 John Wiley 儿 Sons, Inc. 
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Table 1.1 Commonly Used Commands and Functions in MATLAB 


plot(y) (y: read-valued) plot the values of vectorNarray over the index 


plot(y) (y: complex-valued) “plot the imaginary part versus the real part: 














plot(Greal(y)imag(y)) 
bar(y，S1S2S3) The string of three characters S1S?S3，given as one of the 
plot(y， sl1S2S3) input arguments to these graphic commands specifies the 
color, the Symbol, and the line types: 
Stairs(y，S1S2S3) S1(color): y(ellow), m(agenta)，c(yan), r(ed)，g(reen)， 
Stem(y，S1S2S3) bdue)，w(hite),， (blacjk 
loglog(y, S1S?S3) S2(Symbol):.(point)，o,X, 十 ,#,，S(quare: 口 ), diamond:O)， 
Semilogx(y，S1S2S3) V(V)，(A),，<(<),，>(>), p(entagram: 交 ), h(exagram) 
Semilogy(y,，S1S2S3) Ss3(line Symbol): -(Solld, defaultb)，:(dotted)， 
plot(y1, sl1S2S3,，y2, S1S2S3) -.(dashdob),-(dashed) 


(ex) plot(x,，'b+:') plots xn) with the 十 Symbols on 
a blue dotted line 


polar(theta.:T) plot the graph in polar form with the phase theta and 
Imagnitude 


4UXILiary G1apjic Co7a211201205 
axis([xmin xmax ymin ymax]) Specify the ranges of graph on horizontal/vertical axes 


clf(clear fsgure) clear the existent graph(S) 

grid on/o 娃 draw/remove the grid lines 

hold on/off keep/remove the existent graph(S) 

SubplotGjk) divide the Screen into ji x j sections and use the kh one 
text(X,y;,Plot(y， 兴 兴 关 ) print the String“*###x” in the position (X,y) on the graph 
title( # 半 > )，Xlabel( 尖 )， Print the String“##” into the top/low/left Side of graph 


ylabel(“*#*7) 
7jjee-Dimze11sio11aL C1aDHic Co11211201105 


mesh(X,Y, Z) connect the points of height Z at points (X,Y) where 
X,Y and Z are the matrices of the same dimension 


Imesh(x, y, Z) connect the points of height ZU, D at points Specified by 
the two vectors (X(G),yO)) 

Imesh(Z), surfO, plot3()， connect the points of height ZU, i) at points Specified by 

contourO) dj) 


Once you installed MAILAB, you can click the icon like the one in the left Side 
to run MAILAB. Then you will see the MAILAB command window 

了 on your monitor as deplicted in Fig. I.1，where a_ cursor appears 
二 (most 1likely blinking) to the right of the prompt like “>>”oOr 

LU 2 waiting for you to type in a command. 开 you are running 
MAILAB of version 6.x, the main window has not only the command window， 
but also the workspace box and the command history box on the left-up/down 
Slide of the command window, in which you can see the contents of MAILAB 









忆 MATLAB 3 =| 口 | 广 | 
File Edit Siew web 风 indow Help 
区 | 部 恤 色 wx [3 董 有 if | cumert Directory: |cwarLaBepswork = 了 













Workspace 引 | command windo 


严 归 =| 夯 ‖ 咱 Usine Toolbox Path Cache， Type “help mm { 医 





Nane To get started，select “ITLAB Help”from the Help menu， 


Figure 1.1 The MATLAB command window with the workspace box and the command box. 


memory and the commands you have typed into the Command window up to 
the present time，respectively. You might clear the boxes by clicking the cor- 
IeSsponding Submenu under the “Edit” menu and even remove/restore them by 
Un-checking/checking the corresponding Submenu under the“View”menu. 

How do we work with the MAILAB command window?7 


By clicking “File” on the top menu and then New /Open in the File pull- 
down menu, you can create/edit any file with the MAITLAB editor. 


By clicking “File” on the top menu and then Set Path in the File pull-down 
Imenu,， you can make the MAILAB search path include/exclude the paths 
containing the files you want to be run. 

Hyou are abeginner in MAILAB, then it may be worthwhile to click Help” 
on the top menu, click “Demos” in the Help pull-down menu, (double-)click 
any topic that you want to learn, and watch the visual explanation about it. 
By typing any MAILAB commands/statements in the MAILAB command 
window，you can use various powerful mathematic/graphic functions 
of MAILAB. 

H you have an m-file that contains a Series of commands/statements com- 
posed for performing your job, you can type in the fle name (without the 
extension“.m") to make it run. 


It ls helpful to know the procedure of debugging im MAILILAB，which is 
Summarized below. 


1.， With the program (you want to editb) loaded into the MAILAB Editor/ 
Debugger window，set breakpoint(s) at any statement(S) which you think 
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Figure 1.2 The MATLAB file editordebugger window. 


jls (are) Suspicious to be the Source(S) of error,， by clicking the pertinent 
statement line of the program with the left mouse button and pressing the 
El12 key or clicking “SetClear Breakpoint in the “Breakpoints” pull-down 
menu of the EditovDebugger window. Then, you will see a Small red disk 
in front of every statement at which you set the breakpoint. 


. Going to the MAILAB Command window, type in the name of the file 


containing the main program to try running the program. Then， go back to 
the EditorDebugger window and you will see the cursor blinking just after 
a green arrow between the red disk and the first statement line at which 
you Set the breakpoint， 


. Determining which variable to look into，go to the Command window 


and type in the variable name(S) (ust after the prompt “K>>”) Or whatever 
statement you want to Tun for debugging. 


. 开 you want to proceed to the next statement line in the program，go back 


to the EditovDebugger window and press the F10 (Single_step) key or the 
Fl11 (Step_in) key to dig into a called routine. If you want to jump to the 
next breakpoint, press F3 or click Run (Continue) ”in the Debug pull-down 
Imenu of the EditovDebugger window. 玉 you want to run the program until 
just before a statement，move the cursor to the line and click“Go Until 
Cursor in the Debug pull-down menu (See Fig. I.2). 


. 开 you have figure out what is wrong, edit the pertinent part of the program， 


Save the edited program in the EditorDebugger window, and then go to the 
Command window, typing the name of the file containing the main program 
to try running the program for test. 开 the result Seemas to reflect that the 
program Still has a bug, go back to step 1 and restart the whole procedure. 
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HI you use the MAILAB of version 3$.X,，you can refer to the usage of the 
constrained minimization routine“constr() ，which is summarized in the box 
below. 


USAGE OF ITHE MATILAB S.X BUILTI-IN FUNCTIION “CONSTR()7” 
FOR CONSTRAINED OPTIMUZATION 


[x,options] = constr('ftn',x0,options,1,U) 


。JInput arguments (only “ftn” and x0 Tequired, the others optional) 


ftn” :usually defined in an m-file and Should return two output 
arguments, one of which is a scalar value ( 太 GX)) of the 
function (ftn) to be minimized and the other is a vector 
(g(CX)) of constraints Such that g(CX) < 0. 

X0 : the initial guess of solution 

options: is used for setting the termination tolerance on X， 三 (X)，and 
constraint violation through options(2)/(3)/(4), the number of 
the (leading) equajlity constraints among g(CX) < 0 through 
options (13)，etc. 
(For more details, type “help foptions” into the MAILAB 
command window) 

1U : lowerupper bound vectors Such that 1 < X < UL 


。 Output arguments 


X : minimum point reached in the permissible region Satisfying 
the constraints. 

options: outputs Some information about the Search process and the 
Tesult like the function value at the minimum point (X) 
reached through options (8). 
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人 


Absolute error, 33 

Acceleration of Aitken, 201 

Adams-Bashforth-Moulton (ABM) method， 
209 

Adaptive input argument, 40 

Adaptive quadrature, 231 

Alignment,，30 

alternating direction implicit (ADD method， 
417 

Animation，302,，438 

Apostrophe,，15 

Approximation，124, 209, 212, 323 


也 


backslash,，19, 59, 60, 76, 109, 110 

backward difference approximation, 210 

backward substitution，82 

basis function，420 

bilinear interpolation，142 

bisection method,，183 

Boltzmann,， 3335 

boundary condition，134, 401, 404, 420， 
430--432 

Boundary mode, 434 

boundary node, 420 

boundary value problem (BVP), 287, 305-319 

bracketing method,， 188 

breakpoint，493 

Bulirsch- Stoer，161 


人 G 


case,， 24 

catastrophic cancellation,，32 

central difference approximation, 211, 212 
characteristic equation,，371, 465 


characteristic value, 371, 465 
characteristic vector, 371, 465 

Chebyshev coefficient polynomial，120 
Chebyshev node, 123, 160 

Chebyshev polynomial，124, 127, 240 
chemical reactor, 297 

Cholesky decomposition (factorization)，97 
circulant matrix,，391 

conjugate gradient，332 

constrained linear least Squares (LLS), 354 
constrained optimization，343, 330, 352 
constructive Solid geometry (CSG), 432 
contour，11, 295$, 345,， 349 
convergence,，103, 378-379 

covariance matrix,， 386 

Crank-Nicholson method, 409, 452 

CUET, 68, 475 

cubic Spline,，133，102--164 

curve fitting，143,，147，165，1067 


D 

damped Newton method,，193 

data file, 47 

dat-file, 2 

dc motor, 298 

debugging, 493 

decoupling, 374, 370 

determinant，404 

DFT, 191-1$6, 171-175 

diagonalization，374-376 

difference approximation,， 209, 211, 216, 218 

differential equation，2063, 487 

Dirichlet boundary condition，404, 430, 434， 
452 

discretization, 281 

distinct eligenvalues,， 373 

divided difference,，120--122 


4PPlied Nu1aerical Metpods Using 1M4774B9， by Yang, Cao, Chung, and Morris 


Copyright @ 2005 John Wiley 儿 Sons, Inc. 


499 


500 SUBJECT INDEX 


double integralintegration,， 241, 259 
Draw mode, 432 


DtUFT (Discrete-tme Fourier Transform), 476 


玉 


elgenmode PDE, 431 

eligenpair，371 

elgenvalue, 371, 377, 385, 389, 465 
eligenvalue problem, 314, 389 
elgenvector, 371, 377, 385, 465 
electric potential, 427, 442 
element-by-element operation，15, 52 
elliptic PDE, 401, 402, 420, 430 
eps, 13 

error, 31, 33, 39, 40, 213, 226, 274 
error analysis，139, 225 

error estitmate, 220 

error magnification，31 

error propagation,， 33 

erITrorbar，148 

Eulers method, 263 


explicit central difference method, 4153, 417 
explicit forward Euler method, 406, 410 


exponent field, 28 


工 


factoriaL, 40 

false position，185 

FEFT (Fast Fourier Transform)，151 
finite difference method (EDMD), 290 


finite element method (FEM), 420, 431, 455 


fxed-point，99，179,，197 
Fletcher-- Reeves (FR), 332, 333 


forward difference approximation， 209, 218， 


406 
Fourier series/transform，1530, 475 
full pivoting，85 


G 


Gauss elimination，79 

Gauss quadrature,， 234 
Gauss-Chebyshev,，240 
Gauss-Hermite, 238, 291, 253 
Gauss-Jordan elimination，89，1006 
Gauss-Laguerre, 239, 234, 255 
Gauss-Legendre, 233, 251, 255 
Gauss-Seidel lteration，100,，103,，115 
Gaussian distribution，24 

genetic algorithm,，338, 340 
Gerschgorin s Disk Theorem, 380 
golden search, 321, 322 

gradient, 294, 328, 330, 471 
graphic command, 491 


下 


Hamming method, 273 

heat equation, 406, 412 

heat fow equation, 410 
Helmholtz's equation, 402 
Hermite interpolating polynomial 139， 
Hermite polynomial,，66, 238 
Hermitian, 4606 

Hessenberg form, 395--397 
Hessian,， 330, 472 

Heun's method,， 2060 

hidden bit 28 

Hilbert matrix，88 
histogram,，23 

Householder, 392--395 


hyperbolic PDE, 401, 414, 430, 440, 453 


工 


IDFT, 151 

IEEE 64-bit foating-point number,， 28 
ilaplace, 280 

ill-condition，88 


implicit backward Euler method, 407, 452 


improper integral, 248,，249 

inconsistency，83，85 

independent eligenvectors,， 373 

input，2,4 

interlor node, 425 

interpolation，117，119, 133,，141,， 101 
2-dimenslional 141 

interpolation by using DFS, 155 

interpolation function, 420 

inverse matrix，92, 465 

inverse power method, 380, 381 

IVP (initial value problem), 263, 284 


可 


Jacobl iteration，98 
Jacobl method, 381--384 
Jacoblan，191, 472, 484 


开 
keyboard input,， 2 


工 
Lagrange coefficient polynomial，118 


Lagrange multiplier method,，74, 343, 344 


Lagrange polynomial, 117,， 118 
Laguerre polynomial,， 239 
Laplace transform,，278, 280, 473 


Laplace"s Equation, 402, 404, 427, 433, 442 


largest number in MAILAB, 27 
leakage, 155，174 


least Squares (LS), 144, 163, 169, 171, 331, 354 
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Legendre polynomial, 230 卫 
length of arc/curve, 257 Pade approximation，129, 160 
limit 483 parabolic PDE, 406, 410, 412, 414, 430, 438， 
linear equation，71,，79 449 
linear programming (LP), 355, 361 two-dimensional PDE, 412 
logical operator,， 25 parallelepiped, 389 
loop, 20 parallelogram, 388 
loop iteration,，39 parameter passing through VARARGIN, 45 
Lorenz equation，297 parameter sharing via GLOBAL, 44 
loss of significance,，31,，32 partial differential equation (PDE), 401 
LSE (least squares errorD)，75 partial pivoting，81,85,105 
LU decomposition (factorization)，92 path,，1 
PDE mode, 434 
M PDEtool, 429-431, 435, 456 
mantissa field, 28 penalty, 346-349, 362, 360 
mat-fle， 2 permutation，94, 467 
mathematical functions，10 persistent excitation，169 
Imatrix，13, 463 physical meaning of eligenvalues and 
matrix inversion lemma,， 78, 469 elgenvectors, 385 
mean value theorem, 461 pivoting,， 85-88，105-106 
Imesh，11, 48, 49, 431--444 plot, 6-11 
midpoint rule, 222 Plot mode, 440 
minimum-norm solution，73 Polak-Ribiere (PR) method, 332, 333 
Imixed boundary condition， 287, 306, 308 polynomial approximation，124 
modal matrix, 373--376 polynomial curve fitting by least Squares，140， 
mode, 2853, 377-378, 386, 432, 434 169 
modification formula, 272, 274 polynomial wiggle，124 
Imu law, mu-inverse law, 93, 335 positive definite, 468 
Predictorcorrector errors, 272 
N Projection operator，74 
negligible addition, 31 pseudo (generalized) inverse,，17,，73, 76 
Nelder-Mead Algorithm, 325 
nested computing,，38，121 Q 
nested (calling) routine, 40 QR decomposition (factorization),，97, 392-390 
Neumann boundary condition, 404, 431, 441， quadratic approximation method,，323 一 325 
448, 451 quadratic interpolation，157 
Newton method,，186, 188, 191, 330, 332 quadratically convergent，188 
Newton polynomial，119 quadrature, 222, 231, 234 
nonlinear BVP, 312 quantization error,， 63, 212 
nonlinear least Squares (NLLS), 352 quenching factor, 335 
nonnegative least Squares (NLS), 355 
norm,， 58 及 
normal (Gaussian) distribution，24 Tank, 467 
normalized range, 29 Tecursive, 40,，66, 176, 201, 228, 231 
null Space，73,， 467 tecursive least Square estimation (RLSE)，70， 
numerical differentiation，209, 244 104 
numerical integration，222, 247, 249 redundancy, 83, 85 
regula falsi,， 185 
O Telational operators,，25 
on-line recursive computation of DFT, 176 Telative error,，33 
orthogonal, 382, 393, 466 Telaxation，104, 115 
orthonormal,，382, 385 Ieserved constants/variables，13 
Over-determined，75S Richardson'"s extrapolation, 211, 216 


overflow, 34, 64 RLSE (Recursive Least Squares Estimation)，70 


502 SUBJECT INDEX 


Tobot path planning，164 
Romberg integration,， 228-230 
Totation matrix,， 382,，384 
TIound-off error, 31, 3$, 212, 213 
TOow echelon form, 467 

IOW Space, 467 

IOow Switching, 91,，105 

Runge Phenomenon，124 
Runge-Kutta (RK4),，207 
Tuntime error, 40 


S 


Saddle point, 358 

Sampling period,，151,， 153，172 

SCcalar product，4069 

Scaled partial pivoting,，85,，105 

Scaled power method, 378--379 

Schroder method, 202 

Secant method,，189, 201 

Self-calling, 201 

Shifted inverse power method, 380 

Shooting method, 287, 303, 309, 312 

Shooting position，288, 307 

Similarity transformation，373 

Simpson' s rule, 222, 220 

Simulated annealing, 334, 336 

Sinc function, 41, 51 

Single_step, 494 

Smallest positive number in MATILAB， 
27 

Solve mode, 434 

SOR (Successive over-Telaxation)，104 

Sparse, 489 


Stability,，378, 386, 406-410, 415-4106, 418， 


450 
state equation,， 277, 281, 283, 293, 299 
Steepest descent, 328, 330 
Steffensen method, 201 


step-Size, 212--215, 204--265,， 209, 286, 328， 


332 
step-Size dilemma, 213 
step_in, 494 


stiff, 284--286,， 298--299, 3806 
Sturm-Liouville (BVP) equation, 319 
Surface area of revolutionary object, 258 


SYD (Singular value decomposition), 98，112 


Symbolic,，193, 233, 280, 481 
Symbolic variable，194, 481 
Symmetric matrix, 381--382, 466 
Symmetric Diagonalization Theorem, 382 


工 

Taylor series theorem, 462, 485 
temperature, 404, 406, 412, 435,， 438 
term-wise operation，13, 52 

Toeplitz matrix,，390 

trapezolidal rule, 222, 225, 226 
tri-diagonal,，107，108 

truncation error, 31, 212, 213 
two-dimensional interpolation，141,， 168 


U 

unconstrained optimization,，321, 350 
unconstrained least Squares, 355 
underdetermined，72 

underflow, 34, 64 

uniform probabilistic distribution，22 
unitary, 466 

un-normalized range, 29 


V 

Van der Pol equation, 285, 296 
vector，13, 4069 

vector differential equation,， 277, 284 
vector operation，39 

vector product,， 469 

vibration, 416, 418, 440 

volume, 243, 2538 


W 
wave equation, 414, 416-418, 453 


weight least-squares (WLS), 145,， 147,，171 


入 
Zero-padding，151 


INDEX FOR MATLAB 
PROUTINES 





(cf) A/C/E/P/SAT stand for Appendix/ChaptewExample/Problems/Section/Table, respectively. 
(cf) The routines whose name starts with a capital letter are constructed in this book. 
(cf) A _ program named “nmijk.m”can be found in Section 1.]-K. 
abmoc S6.4-1 PredictorCorrector coefficients in Adams-Bashforth- 
Moulton ODE solver 
adapt_SmpsnO) S3.8 ntegration by the adaptive Simpson method 
adc10) P1.10 AD conversion 
adc20) P1.10 AD conversion 
axis(O S1.1-4 Specify axis limits or appearance 
backslash(A) P1.14 left matrix division 
backsubst(O) S2.4-1 backward substitution for lower-triangular matrix 
equation 
barO/barhO S1.1-4 a vertical/horizontal bar chart 
bisctO S4.2 bisection method to solve a nonlinear equation 
break S1.1-9 terminate execution of a for loop or while loop 
bvp2_eigO P6.11 Solve an eligenvalue BVP2 
bvp2_fdfO S06.6-2 FDM (Finite difference method) for a BVP 
bvp2_fdfpO P6.6 FDM for a BVP with initial derivative fxed 
bvp2_shootO S6.60-1 Shooting method for a BVP (boundary value problemy) 
bvp2_shootpO P6.6 Shooting method for a BVP with initial derivative fxed 
bvp2_fdfO S06.6-2 FDM (Finite difference method) for a BVP 
bvp2_fdfpO P6.6 FEDM for a BVP with initial derivative fxed 
bvp2m_shootpO P6.7 Shooting method for BVP with mixed boundary 
condition 工 
bvp2m_fdfpO P6.7 FEDM for a BVP with mixed boundary condition 工 
bvp2mm shootpO P6.8 Shooting method for BVP with mixed boundary 
condition 开 
bvp2mm_ fdfpO P6.8 FEDM for a BVP with mixed boundary condition 工 
bvp2_fdfpO P6.6 Finite difference method for a BVP with inltial 
derivative 
bvp4cO S6.6-2， fxed 
P6.7~10 BVP solver 
cell() S1.1-3 (T1.3) | round toward infinlity 
chebyO S3.3 Chebyshev polynomial approximation 
cholO S2.4-2 Cholesky factorization 
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clear S1.1-2 Temove items from workspace, freeing up System 
memory 
clf S1.1-4 clear current fgure window 
compare_DFT_FEFT S3.9-1 compare DFT with FFT 
condO S2.2-2 condition number 
Constr(O) AH constrained minimization (in MAILAB 53.X) 
contour(O) S1.1-9 2-D contour plot of a scalar-valued function of 2-D 
variable 
conVO S1.1-6 convolution of two sequences 
or multiplication of two polynomials 
cspline(O) S3.5 cubic spline interpolation 
CtFT10 P1.20 Inverse Continuous-time Fourier Transform 
curve_fitO P3.9 weighted least-squares curve fitting 
c2d_steqO) S06.5-2 continuous-time state equation to discrete-time one 
dblquad(O S1.1-7 2-D (double) integral 
diagO S5.3 construct a diagonal matrix or get diagonals of a matrix 
difapx(O) S5.4, AG2-3 | difference approximation for numerical derivatives 
diffO 8S5.10, P3.14 | differences between neighboring elements in an array 
dispO S1.1-3 display text or array onto the (monitor) Screen 
do_cheby S3.3 approximate by Chebyshev polynomial 
do_condition S2.2-2 condition numbers for ill-conditioned matrices 
do_csplines S3.5 interpolate by cubic splines 
do_FFT S3.9-1 do FFT (Fast Fourier Transform) 
do_gauss S2.2-1 do Gauss elimination 
do_hermit S3.0 do Hermite polynomial interpolation 
do_interp2 S3.7 do 2-dimensional interpolation 
do_lagranp S3.1 do Lagrange polynomial interpolation 
do_lagnewch S3.3 try Lagrange/Newton/Chebyshev polynomial 
do_lu_dcmp S2.4-1 do LU decomposition (factorization) 
do_MBK P06.4 Simulate a mass-damper-Spring System 
do_newtonp S3.2 do Newton polynomial interpolation 
do_newtonp1l S3.2 do Newton polynomial interpolation 
do_pade(O) S3.4 do Pade (rational polynomial) approximation 
do_polyfitsO S3.8-2 do polynomial curve fitting 
do_RDFT P3.20 do recursive DFT 
do_quiver P6.0 Use quivergO to plot the gradient Vectors 
do_rlse S2.1-4 do recursive least-squares estimation 
do_wlse S3.8-2 do weighted least-squares curve fitting 
doubleO) AG1 convert to double-precision 
draw_MB 开 P06.4 simulate a mass-damper-Spring System 
dsolveO) S0.0-2， Symbolic differential equation Solver 
P6.3, AG5 
elg(O) S8.1 elgenvalues and eligenvectors of a matrix 
eig_Jacobi0 S8.4 find the eligenvalues/eigenvectors of a Symmetric matrix 
elg_power(O) S8.3 find the largest eigenvalue & the corresponding 
elgenvector 
elg_QRO P8.7 fnd eligenvalues using QR factorization 
elg_QR_HsO P8.7 fnd eligenvalues using QR factorization via Hessenberg 
else S1.1-9 for conditional execution of statements 
elseif S1.1-9 for conditional execution of statements 
end S1.1-9 terminate for/while,/witch/try/if statements or last index 
err_of Sol _deO P6.9 evaluate the error of solution of differential eq. 
eval(O S1.1-3 (T1.3) | evaluate a string containing a MAILAB expression 





eye 

ezplot(O 

falspO 
fem_basis_ftn(O) 
fem_coefO 
feval0: 


fndO 
fndsymO 
fxO) 

fxptO) 
fiplrO 

位 pudO 
foorO) 
fminbnd(O) 
fminconO) 
fminimax(O) 
fminsearch() 
fminuncO 
for 

format 
forsubstO 
fprintfO 
fsolveO) 
gauseld() 


gauss(O) 
gauss_legendre(O 


gausslp(O) 
gausshp(O) 
geneticO 
ginputO 


global 
gradientO 
grid on/off 


gtext(O) 
heat_expO) 


heat_impO 


heat_CNO 
heat2_ADIO 
help 
hermitO 
hermitp() 
hermitsO) 
hessenbergO 
histO 

hold on/off 
housholderO 
ICtET10 

让 


S1.1-7 
91.3-0 
34.3 
39.4 
39.4 
S1.1-0 


P1.10 
S4.7 

S1.1-5 (T1.3) 
S4.1 

S1.1-7 

S1.1-7 

S1.1-5 (T1.3) 
S7.1-2 
S7.3-2 
S7.3-2 
S$7.222,.73 
S7.2-2, 7.3-1 
S1.1-9 

S1.1-3 

S2.4-1 
S1.1-3, P1.2 
S4.6,4.7.E4.3 
S2.5-2 


S2.2-2 
935.9-1 
95.9-1 
S5.9-2 
37.1-8 
91.1-4 


S1.3-5 
P6.0 

91.1-4 
91.1-4 
939.2-1 


39.2-2 


39.2-3 
39.2-4 
91.1-1 
9S3.0 
35.9-2 
93.0 
P8.5 
S1.1-4,，1.1-8 
S1.1-4 
P8.4 
P1.26 
91.1-9 
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identity matrix (having 1/0 on/off its diagonal) 

easy plot 

false position method to Solve a nonlinear equation 

coefficients of each basis function for subregions 

coefficients for Subregions 

evaluation of a function defined by inlineO or in an 
M-file 

fnd indices of nonzero (true) elements 

find the Symbolic variables in a Symbolic expression 

TIound towards zero 

fxed-point iteration to Solve a nonlinear equation 

fip the elements of a matrix left-right 

fip the elements of a matrix up-down 

Iound to 一 infinity 

unconstrained minimization of one-variable function 

constrained minimization 

Iminimize the maximum of vectormatrix-valued function 

unconstrained nonlinear minimlization (Nelder-Mead) 

unconstrained nonlinear minimization (gradient-based) 

Tepeat Statements a Specific number of times 

control display format for numbers 

forward substitution for lower-triangular matrix equation 

WwWrIite formatted data to Screen or file 

Solve nonlinear equations by a least Squares method 

Gauss-Seidel method to solve a System of linear 
equations 

Gauss elimination to Solve a System of linear equations 

Gauss-Legendre integration 

grid points of Gauss-Legendre integration formula 

grid points of Gauss-Hermite integration formula 

optimization by the genetic algorithm (GA) 

input the x- 贸 y-coordinates of point(S) clicked by 
Imouse 

declare global variables 

numerical gradient 

grid lines for 2-D or 3-D graphs 

Imouse placement of text in a 2-D graph 

explicit forward Euler method for parabolic PDE (heat 
eq) 

implicit backward Euler method for parabolic PDE (heat 
eq) 

Crank-Nicholson method for parabolic PDE (heat ed) 

ADI method for parabolic PDE (2-D heat equation) 

display help comments for MAITLAB routines 

Hermite polynomial interpolation 

Hermite polynomial 

multiple Hermite polynomial interpolations 

transform a matrix into almost upper-triangular one 

plot a histogram 

hold on/off current graph in the fgure 

Householder matrix to zero-out the tail part of a vector 

JInverse Continuous-time Fourier Transform 

for conditional execution of statements 
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inlineO 
inpolygon(O) 
inputO 

intO 
interp10 
interp20 
intrp10 
intrp20) 


interpolate_by_DFS 


int2sO 

invO 
isemptyO) 
lsnumericO 
jacobO 
jacob10 
jacobi0 
JkbO 
lagranp(O 
lgndrpO 
lengthO 
limitO 
lin_eqO) 
linprogO 
load 
loglogW 
lookfor 
lscovO 
lsqcurvefitO 
lsqlin() 
lsqnonlin() 
lsqnonnegO 
lu0 
lu_dcmpO 
max(O 
mesh(O) 
Imeshgrid(O) 
min(0) 
mkppO 
modO 
mulawO) 
mu _invO) 
multiply_matrix(O) 
newton() 
newtonp(O) 
newtons() 
norm() 
ode _ABMO 


ode_EulerO) 
ode_HamO 
ode_HeunO) 
ode_RK40 
ode23(0/ode450 


SL16 
S9.4 
S1.1-3 
S5.8, AG2 
S3.5 

S3.7 

P3.10 

S3.7 
S3.9-3 
S5.10, P5.14 
S1.1-7 
P1.10 
P1.10 

S4.6 

P5.3 
S2.5-1 
P1.21 

S3.1 
S5.9-1 
S1.1-7 
AG2-2 
S2.1-3 
S7.3-3 
S1.1-2,4 
S1.1-4 
S1.1-1 
S3.8-1 
S3.8-3 
S7.3-1 
S7.3-1 
S7.3-2 
S2.4-1 
S2.4-1 
S1.1-7 
S1.1-5, 3.7 
S1.1-5, 3.7 
S1.1-7 
P1.11 
S1.1-5 (T1.3) 
P1.9 
S7.1-7 
P1.12 

S4.4 

S3.2 

S4.6 

P1.13 
S6.4-1 


9S0.1 
30.4-2 
S0.2 
S0.3 
S0.4-3 


define a function inside the Program 

is the point inslde an polygonal region2? 

Tequest and get user input 

numerical/symbolic integration 

1-D interpolation 

2-D interpolation 

1-D interpolation 

2-D interpolation 

interpolation using DFS 

2-D (double) integral 

the inverse of a matrix 

is it empty (no value)7 

has it a numeric value? 

Jacobian matrix of a given function 

Jacobian matrix of a given function 

Jacobi iteration to Solve a equation 

1s kind of 人 -th order Bessel function 

Lagrange polynomial interpolation 

Legendre polynomial 

the length of a vector (Sequence) 

limit of a Symbolic expression 

Solve linear equation(S) 

Solve a linear programming (LP) problem 

Tead variable(s) from file 

plot data as logarithmic Scales for the x-axis and y-axis 

Search for string in the first comment line in all M-files 

weighted least-squares with known (error) covariance 

weighted nonlinear least-Squares curve fitting 

Solve a linear least Squares (LLS) problem 

Solve a non-linear least Squares (NLLS) problem 

find a non-negative least Squares (NNLS) solution 

LU decomposition (factorization) 

LU decomposition (factorization) 

fnd the maximum element(s) of an array 

plot a mesh-type graph of fx, y) 

grid points for plotting a mesh-type graph 

fnd the minimum element(s) of an array 

Imake 3a piece-wise polynomial 

TIemainder after division 

AM-law 

HL law 

matrix multiplication 

Newton method to solve a nonlinear equation 

Newton polynomial interpolation 

Newton method to Solve a System of nonlinear equation 

norm of vectormatrix 

Solve a state equation by Adams-Bashforth-Moulton 
SOlver 

Solve a state equation by Eulers method 

Solve a state equation by Hamming ODE solver 

Solve a state equation by Heun's method 

Solve a state equation by Runge-Kutta method 

ODE solver 





/ode1130 
ode1SsO/ode23SO 
/ode23t(O/ode23tb(O) 
ones() 

opt_gsO 
opt_quad0O) 
opt_NelderO 
opt_steepO 
opt_conjg0) 
padeap0) 

pdetool 

pinvO 

plotO 

plot30) 

poisson(O 

polar0 


Poly_derO 
polyderO 
polyfitO 
PolyfitsO) 
polyintO 
polyval0O 
ppvalO 
pretty0) 
prod() 

drO 
qdTr_hessenpbergO) 
quad() 
quadl0 
duiverO 
duiver30) 
Iand() 
Tandn(O) 
rational _interpolation() 
repetition() 
Teshape(O 
Tesidue(O 


TesiduezO) 


Ilse_onlineO) 
ImbrgO 
Iobot_path 
Toots(O) 
TIound() 
rot900) 
Save 
SecantO 
Semjlogx() 
Semjlogy(O) 
SiZze(O) 
Sim_anl(O) 


930.5-4 


S1.1-7 
S7.1-1 
S7.1-2 
97.1-3 
937.1-4 
S7.1-0 
9S3.4 
39.4 
S1.1-7, 2.1 
31.1-4,5 
S1.1-9 
939.1 
91.1-4 


P1.11 
P1.11 
P3.13 
33.8-2 
P1.11 
91.1-0, 3.8-2 
P1.11 
P3.1, AG2 
91.1-7 
S2.4-2 
P8.0 
S5.8 
S5.8 
P6.0 
P6.0 
S1.1-8 
S1.1-8 
P3.0 
P1.14 
91.1-7 
P1.11 


P1.11 


S2.1-4 
S5.7 

P3.9 

P1.11 

S1.1-5 (T1.3) 
S1.1-7 
S1.1-2 

S4.5 

S1.1-4 
S1.1-4 
S1.1-7 

S7.1-7 
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Solve (Stiffb ODEs 


constructs an array of ones 

optimization by Golden search 

optimization by quadratic approximation 

optimization by Nelder-Mead method 

Optimization by Steepest descent 

optimization by Conjugate gradient method 

Pade approximation 

start the PDE toolbox GUI (graphical user interface) 

pseudo-inverse (generalized inverse) 

linear 2-D plot 

linear 3-D plot 

central difference method for elliptic PDE (Poisson s eq) 

plot polar coordinates in a Cartesian plane with polar 
grid 

derivative of polynomial 

derivative of polynomial 

polynomial curve fitting 

polynomilal curve fitting 

integral of polynomial 

evaluate a polynomial 

evaluate a Set of piece-wise polynomials 

Print Symbolic expression like in type-set form 

product of array elements 

QR factorization 

QR factorization of Hessenberg form by Givens rotation 

numerical integration 

numerical integration 

plot gradient vectors 

plot normal vectors on a Surface 

uniform random number generator 

Gaussian random number generator 

rational polynomial interpolation 

Tepetition of Subsequences 

a matrix into one with given numbers of row/columns 

partial fraction expansion of Laplace-transformed 
function 

partial fraction expansion of z-transformed rational 
function 

on-line Recursive Least-Squares Estimation 

JIntegration by Romberg method 

determine a path of robot using cubic splines 

roots of a polynomial equation 

Iound to nearest integer 

Totate a matrix by 90 degrees 

Save variable(s) into a file 

Secant method to Solve a nonlinear equation 

plot data as logarithmic Scales for the x-axis 

plot data as logarithmic Scales for the y-axis 

the numbers of rows/columns of a 1-D/2-D/3-D array 

optimization by Simulated annealing (SA) 
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Simple(O) AG2-3 simplest form of symbolic expression 
SimplifyO AG2-3 Simplify Symbolic expression 
Smpsns(O) S5.0 Integration by Simpson rule 
Smpsns_fxyO S5.10, P5.15 | 1-D integration of a function Fx, y) along y 
Solve(O P3.1，S4.7， Symbolic solution of algebraic equations 
AG4 
Sort() S1.1-4 arranges the elements of an array in ascending order 
splineO S3.5 cubic spline 
SprintfO S1.1-4 make formatted data to a String 
Stairs(O) S1.1-4 stair-Step plot of zero-hold Signal of sampled data 
Systemas 
stem(O) S1.1-4 plot discrete sequence data 
SubplotO S1.1-4, 1.1-7 | divide the current figure into rectangular panes 
SubsO AG1 Substitute 
Sum() S1.1-7 Sum of elements of an array 
Surface() P6.0 plot a Surface-type graph of 太 x, y) 
Surfnorm() P6.0 generate vectors normal to a Surface 
SVd(O) S2.4-2 Singular value decomposition 
Switch S1.1-9 Switch among Several cases 
SyYImS P3.1，9S4.7， declare Symbolic variable(S) 
AG 
Sym2poly(O) S3.3，AG2 extract the coefficients of Symbolic polynomial 
expression 
taylor(O) S3.3，AG2 Taylor series expansion 
text(O) S1.1-4 add a text at the specified location on the graph 
titleO) S1.1-4 add title to current axes 
trid(O) S0.0-2 Solve a tri-diagonal System of linear equations 
trimesh() 39.4 plot a triangular-mesh-type graph 
trpzds(O) S5.0 Integration by trapezoidal rule 
varargin(O) S1.3-0 yariable length input argument list 
viewO S1.1-3, P1.4 13-D graph viewpoint Specification 
VvpaO) AG evaluate double array by Variable precision arithmetic 
wave(O S9.3-1 central difference method for hyperbolic PDE (wave ed) 
Wave20 S9.3-2 central difference method for hyperbolic PDE (2-D 
wave eq) 
while S1.1-9 Tepeat statements an indefinite number of times 
windowingO P3.18 Imultiply a sequence by the Specified window sequence 
XlabelO/ylabel0O S1.1-4 label the X-axXis/y-aXis 
Zeros() S1.1-7 construct an array of zeros 
Zeroing(O) P1.15$ cross out every (kM-m)th element to zero 
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Table 
Table 
Table 
Table 
Table 


Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 
Table 


Table 


Conversion type Specifiers 久 Special characters in fprintfO 

Graphic line Specifications used in the plotO command 

Functions and variables inside MATLAB: 

Relational operators and logical operators 

Residual error and the number of foating-point operations of various 
Solutions 

Divided difference table 


Divided differences 

Chebyshev coefficient polynomial 

Boundary conditions for cubic spline 

Linearization of nonlinear functions by parameter/data transformation 

The forward difference approximation (5.1-4) for the 1st derivative 
and its error depending on the step-Slze 


The forward difference approximation (S.1-8) for the 1st derivative 
and its error depending on the step-Slze 

The difference approximation formulas for the lst and 2nd derivatives 

Rombersg table 

A numerical solution of the differential equation (6.1-1) obtained by 
the Eulers method 

Results of applying Several routines for solving a Simple differential 
equation 

Results of running Several unconstrained optimization routines with 
various initial values 

Results of running several unconstrained optimization routines with 
various initial values 

The names of the MAILAB built-in minimization Toutines in 
MAILAB 5.xX/6.Xx 





(cf) A: Appendix, P: Problem, S: Section, 工 Table 
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